Built on

Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47
2002
Earlier work this paper cites.
Srinivas, N., Krause, A., Kakade, S.M., Seeger, M.: Gaussian process optimization in the bandit setting: No regret and experimental design. arXiv preprint arXiv:0912.3995 (2009)
Original
2009
Earlier work this paper cites.
Li, L., Chu, W., Langford, J., Schapire, R.E.: A contextual-bandit approach to personalized news article recommendation. In: Proceedings of the 19th international conference on World wide web. pp. 661–670 (2010)
2010
Earlier work this paper cites.
Abbasi-Yadkori, Y., Pál, D., Szepesvári, C.: Improved algorithms for linear stochastic bandits. In: NIPS. vol. 11, pp. 2312–2320 (2011)
2011
Earlier work this paper cites.
Krause, A., Ong, C.S.: Contextual gaussian process bandit optimization. In: Nips. pp. 2447–2455 (2011)
2011
Earlier work this paper cites.
Chen, W., Wang, Y., Yuan, Y.: Combinatorial multi-armed bandit: General framework and applications. In: International Conference on Machine Learning. pp. 151–159. PMLR (2013)
2013
Earlier work this paper cites.
Qin, L., Chen, S., Zhu, X.: Contextual combinatorial bandit and its application on diversified online recommendation. In: Proceedings of the 2014 SIAM International Conference on Data Mining. pp. 461–469. SIAM (2014)
2014
Earlier work this paper cites.
Reverdy, P.B., Srivastava, V., Leonard, N.E.: Modeling human decision making in generalized gaussian multiarmed bandits. Proceedings of the IEEE 102
2014
Earlier work this paper cites.

Similar

Kveton, B., Szepesvari, C., Wen, Z., Ashkan, A.: Cascading bandits: Learning to rank in the cascade model. In: International Conference on Machine Learning. pp. 767–776. PMLR (2015)
2015
Cited alongside, same era.
Kveton, B., Wen, Z., Ashkan, A., Szepesvari, C.: Tight regret bounds for stochastic combinatorial semi-bandits. In: Artificial Intelligence and Statistics. pp. 535–543. PMLR (2015)
2015
Cited alongside, same era.
Chen, W., Hu, W., Li, F., Li, J., Liu, Y., Lu, P.: Combinatorial multi-armed bandit with general reward functions. arXiv preprint arXiv:1610.06603 (2016)
Original
2016
Cited alongside, same era.
Degenne, R., Perchet, V.: Combinatorial semi-bandit with known covariance. arXiv preprint arXiv:1612.01859 (2016)
Original
2016
Cited alongside, same era.
Kazerouni, A., Ghavamzadeh, M., Abbasi-Yadkori, Y., Van Roy, B.: Conservative contextual linear bandits. arXiv preprint arXiv:1611.06426 (2016)

Then

Chen, L., Krause, A., Karbasi, A.: Interactive submodular bandit. In: NIPS. pp. 141–152 (2017)
2017
Later among the works it cites.
Boursier, E., Perchet, V.: Sic-mmab: synchronisation involves communication in multiplayer multi-armed bandits. arXiv preprint arXiv:1809.08151 (2018)
Original
2018
Later among the works it cites.
Wang, Z., Zhou, R., Shen, C.: Regional multi-armed bandits with partial informativeness. IEEE Transactions on Signal Processing 66
2018
Later among the works it cites.
Gan, C., Yang, J., Zhou, R., Shen, C.: Online learning with diverse user preferences. In: 2019 IEEE International Symposium on Information Theory (ISIT). pp. 2539–2543. IEEE (2019)
2019
Later among the works it cites.
Zhang, X., Li, S., Liu, W.: Contextual combinatorial conservative bandits. arXiv preprint arXiv:1911.11337 (2019)
Original

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2021

Conservative Contextual Combinatorial Cascading Bandit

Wang, Kun, Zhao, Canzhe, Li, Shuai et al.

Understand

Conservative mechanism is a desirable property in decision-making problems which balance the tradeoff between the exploration and exploitation.

We propose the novel \emph{conservative contextual combinatorial cascading bandit ($C^4$-bandit)}, a cascading online learning game which incorporates the conservative mechanism.
At each time step, the learning agent is given some contexts and has to recommend a list of items but not worse than the base strategy and then observes the reward by some stopping rules.
We design the $C^4$-UCB algorithm to solve the problem and prove its n-step upper regret bound for two situations: known baseline reward and unknown baseline reward.

Built on

Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47
2002
Earlier work this paper cites.
Srinivas, N., Krause, A., Kakade, S.M., Seeger, M.: Gaussian process optimization in the bandit setting: No regret and experimental design. arXiv preprint arXiv:0912.3995 (2009)
Original
2009
Earlier work this paper cites.
Li, L., Chu, W., Langford, J., Schapire, R.E.: A contextual-bandit approach to personalized news article recommendation. In: Proceedings of the 19th international conference on World wide web. pp. 661–670 (2010)
2010
Earlier work this paper cites.
Abbasi-Yadkori, Y., Pál, D., Szepesvári, C.: Improved algorithms for linear stochastic bandits. In: NIPS. vol. 11, pp. 2312–2320 (2011)
2011
Earlier work this paper cites.
Krause, A., Ong, C.S.: Contextual gaussian process bandit optimization. In: Nips. pp. 2447–2455 (2011)
2011
Earlier work this paper cites.
Chen, W., Wang, Y., Yuan, Y.: Combinatorial multi-armed bandit: General framework and applications. In: International Conference on Machine Learning. pp. 151–159. PMLR (2013)
2013
Earlier work this paper cites.
Qin, L., Chen, S., Zhu, X.: Contextual combinatorial bandit and its application on diversified online recommendation. In: Proceedings of the 2014 SIAM International Conference on Data Mining. pp. 461–469. SIAM (2014)
2014
Earlier work this paper cites.
Reverdy, P.B., Srivastava, V., Leonard, N.E.: Modeling human decision making in generalized gaussian multiarmed bandits. Proceedings of the IEEE 102
2014
Earlier work this paper cites.

Similar

Kveton, B., Szepesvari, C., Wen, Z., Ashkan, A.: Cascading bandits: Learning to rank in the cascade model. In: International Conference on Machine Learning. pp. 767–776. PMLR (2015)
2015
Cited alongside, same era.
Kveton, B., Wen, Z., Ashkan, A., Szepesvari, C.: Tight regret bounds for stochastic combinatorial semi-bandits. In: Artificial Intelligence and Statistics. pp. 535–543. PMLR (2015)
2015
Cited alongside, same era.
Chen, W., Hu, W., Li, F., Li, J., Liu, Y., Lu, P.: Combinatorial multi-armed bandit with general reward functions. arXiv preprint arXiv:1610.06603 (2016)
Original
2016
Cited alongside, same era.
Degenne, R., Perchet, V.: Combinatorial semi-bandit with known covariance. arXiv preprint arXiv:1612.01859 (2016)
Original
2016
Cited alongside, same era.
Kazerouni, A., Ghavamzadeh, M., Abbasi-Yadkori, Y., Van Roy, B.: Conservative contextual linear bandits. arXiv preprint arXiv:1611.06426 (2016)

Then

Chen, L., Krause, A., Karbasi, A.: Interactive submodular bandit. In: NIPS. pp. 141–152 (2017)
2017
Later among the works it cites.
Boursier, E., Perchet, V.: Sic-mmab: synchronisation involves communication in multiplayer multi-armed bandits. arXiv preprint arXiv:1809.08151 (2018)
Original
2018
Later among the works it cites.
Wang, Z., Zhou, R., Shen, C.: Regional multi-armed bandits with partial informativeness. IEEE Transactions on Signal Processing 66
2018
Later among the works it cites.
Gan, C., Yang, J., Zhou, R., Shen, C.: Online learning with diverse user preferences. In: 2019 IEEE International Symposium on Information Theory (ISIT). pp. 2539–2543. IEEE (2019)
2019
Later among the works it cites.
Zhang, X., Li, S., Liu, W.: Contextual combinatorial conservative bandits. arXiv preprint arXiv:1911.11337 (2019)
Original

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

2016

Cited alongside, same era.

Li, S., Wang, B., Zhang, S., Chen, W.: Contextual combinatorial cascading bandits. In: International conference on machine learning. pp. 1245–1253. PMLR (2016)

2016

Cited alongside, same era.

Wang, H., Wu, Q., Wang, H.: Learning hidden features for contextual bandits. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. pp. 1633–1642 (2016)

2016

Cited alongside, same era.

Wu, Y., Shariff, R., Lattimore, T., Szepesvári, C.: Conservative bandits. In: International Conference on Machine Learning. pp. 1254–1262. PMLR (2016)

2016

Cited alongside, same era.

2019

Later among the works it cites.

Gan, C., Zhou, R., Yang, J., Shen, C.: Cost-aware cascading bandits. IEEE Transactions on Signal Processing 68

2020

Later among the works it cites.

Garcelon, E., Ghavamzadeh, M., Lazaric, A., Pirotta, M.: Improved algorithms for conservative exploration in bandits. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 34, pp. 3962–3969 (2020)

2020

Later among the works it cites.

Lu, Y., Meisami, A., Tewari, A., Yan, W.: Regret analysis of bandit problems with causal background knowledge. In: Conference on Uncertainty in Artificial Intelligence. pp. 141–150. PMLR (2020)

2020

Later among the works it cites.