To solve convex optimization problems with a noisy gradient input, we analyze the global behavior of subgradient-like flows under stochastic errors. The objective function is composite, being equal to the sum of two convex functions, one being differentiable and the other potentially non-smooth. We then use stochastic differential inclusions where the drift term is minus the subgradient of the objective function, and the diffusion term is either bounded or square-integrable. In this context, under Lipschitz’s continuity of the differentiable term and a growth condition of the non-smooth term, our first main result shows almost sure weak convergence of the trajectory process towards a minimizer of the objective function. Then, using Tikhonov regularization with a properly tuned vanishing parameter, we can obtain almost sure strong convergence of the trajectory towards the minimum norm solution. We find an explicit tuning of this parameter when our objective function satisfies a local error-bound inequality. We also provide a comprehensive complexity analysis by establishing several new pointwise and ergodic convergence rates in expectation for the convex, strongly convex, and Łojasiewicz case.
Revised:
Accepted:
Published online:
Rodrigo Maulen-Soto 1; Jalal Fadili 1; Hedy Attouch 2
CC-BY 4.0
@article{OJMO_2025__6__A10_0,
author = {Rodrigo Maulen-Soto and Jalal Fadili and Hedy Attouch},
title = {Stochastic {Differential} {Inclusions} and {Tikhonov} {Regularization} for {Stochastic} {Non-Smooth} {Convex} {Optimization} in {Hilbert} spaces},
journal = {Open Journal of Mathematical Optimization},
eid = {9},
pages = {1--30},
publisher = {Universit\'e de Montpellier},
volume = {6},
year = {2025},
doi = {10.5802/ojmo.44},
language = {en},
url = {https://ojmo.centre-mersenne.org/articles/10.5802/ojmo.44/}
}
TY - JOUR AU - Rodrigo Maulen-Soto AU - Jalal Fadili AU - Hedy Attouch TI - Stochastic Differential Inclusions and Tikhonov Regularization for Stochastic Non-Smooth Convex Optimization in Hilbert spaces JO - Open Journal of Mathematical Optimization PY - 2025 SP - 1 EP - 30 VL - 6 PB - Université de Montpellier UR - https://ojmo.centre-mersenne.org/articles/10.5802/ojmo.44/ DO - 10.5802/ojmo.44 LA - en ID - OJMO_2025__6__A10_0 ER -
%0 Journal Article %A Rodrigo Maulen-Soto %A Jalal Fadili %A Hedy Attouch %T Stochastic Differential Inclusions and Tikhonov Regularization for Stochastic Non-Smooth Convex Optimization in Hilbert spaces %J Open Journal of Mathematical Optimization %D 2025 %P 1-30 %V 6 %I Université de Montpellier %U https://ojmo.centre-mersenne.org/articles/10.5802/ojmo.44/ %R 10.5802/ojmo.44 %G en %F OJMO_2025__6__A10_0
Rodrigo Maulen-Soto; Jalal Fadili; Hedy Attouch. Stochastic Differential Inclusions and Tikhonov Regularization for Stochastic Non-Smooth Convex Optimization in Hilbert spaces. Open Journal of Mathematical Optimization, Volume 6 (2025), article no. 9, 30 p.. doi: 10.5802/ojmo.44
[1] Topics in Relaxation and Ellipsoidal Methods, Research Notes in Mathematics, 97, Pitman Publishing, 1984 | Zbl | MR
[2] Accelerated gradient methods with strong convergence to the minimum norm minimizer: a dynamic approach combining time scaling, averaging, and Tikhonov regularization (2022) | arXiv
[3] A Dynamical Approach to Convex Minimization Coupling Approximation with the Steepest Descent Method, J. Differ. Equations, Volume 128 (1996) no. 2, pp. 519-540 | DOI | Zbl | MR
[4] Asymptotic behavior of coupled dynamical systems with multiscale aspects, J. Differ. Equations, Volume 248 (2010) no. 6, pp. 1315-1344 | DOI | Zbl | MR
[5] On the effect of perturbations in first-order optimization methods with inertia and Hessian driven damping, Evol. Equ. Control Theory, Volume 12 (2023) no. 1, pp. 71-117 | DOI | Zbl | MR
[6] Quantitative stability analysis for maximal monotone operators and semi-groups of contractions, Nonlinear Anal., Theory Methods Appl., Volume 21 (1993) no. 9, pp. 697-723 | DOI | Zbl | MR
[7] Quantitative stability of variational systems: I, The epigraphical distance, Trans. Am. Math. Soc., Volume 328 (1991) no. 2, pp. 695-729 | Zbl | MR
[8] The viability theorem for stochastic differential inclusions, Stochastic Anal. Appl., Volume 16 (1998), pp. 1-15 | DOI | Zbl | MR
[9] Convex Analysis and Monotone Operator Theory in Hilbert Spaces, Springer, 2017 | DOI | Zbl | MR
[10] On Wiener stochastic integrals of multifunctions, Seminarul de Teoria Probabilitatilor si Applicatii, Univ. Timisoara, 1987, pp. 1-7
[11] From error bounds to the complexity of first-order descent methods for convex functions, Mathematical Programming (Jon Lee; Sven Leyffer, eds.) (Mathematical Optimization Society), Volume 165, Springer, 2016, pp. 471-507 | DOI | Zbl | MR
[12] Operateurs maximaux monotones et semi-groupes de contractions dans les espace Hilbert, North-Holland Mathematics Studies, 5, American Elsevier Publishing Comp., 1973 | Zbl | MR
[13] Existence and approximation of solutions of nonlinear variational inequalities, Proc. Natl. Acad. Sci. USA, Volume 56 (1966), pp. 1080-1086 | DOI | Zbl | MR
[14] Convergence of approximants to fixed points of nonexpansive nonlinear mappings in Banach spaces, Arch. Ration. Mech. Anal., Volume 24 (1967), pp. 82-90 | DOI | Zbl | MR
[15] Integral inequalities for convex functions of operators on martingales, Proc. 6th Berkeley Symp. Math. Statistics and Probability, 1972, pp. 223-240 | Zbl
[16] Méthode générale pour la résolution des systèmes d’équations simultanées, C. R. Acad. Sci. Paris, Volume 25 (1847), pp. 536-538
[17] Équations différentielles stochastiques multivoques, Séminaire de probabilités XXIX (Strasbourg) (Lecture Notes in Mathematics), Volume 1613, Springer, 1995, pp. 86-107 | DOI | Numdam | Zbl
[18] Strong asymptotic convergence of evolution equations governed by maximal monotone operators with Tikhonov regularization, J. Differ. Equations, Volume 245 (2008), pp. 3753-3763 | DOI | Zbl | MR
[19] Stochastic equations in infinite dimensions, Encyclopedia of Mathematics and Its Applications, 44, Cambridge University Press, 1992 | DOI | Zbl | MR
[20] Stochastic Differential Equations in Infinite Dimensions, Probability and Its Applications, Springer, 2011 | DOI | Zbl
[21] A proof and an application of the continuous parameter martingale convergence theorem, Res. Stat., Volume 1 (2023) no. 1 | DOI
[22] Yosida Approximations of Stochastic Differential Equations in Infinite Dimensions and Applications, Probability Theory and Stochastic Modelling, 79, Springer, 2016 | MR | DOI | Zbl
[23] Bibliography on non-differentiable optimization and non-smooth analysis, J. Comput. Appl. Math., Volume 7 (1981), pp. 277-285 | MR | DOI | Zbl
[24] Strong convergence of Euler-type methods for nonlinear stochastic differential equations, SIAM J. Numer. Anal., Volume 40 (2006) no. 3, pp. 1041-1063 | MR | DOI | Zbl
[25] Stochastic differential inclusions and applications, Springer Optimization and Its Applications, 80, Springer, 2013 | DOI | MR | Zbl
[26] Lagrangian relaxation via ballstep subgradient methods, Math. Oper. Res., Volume 32 (2007) no. 3, pp. 669-686 | DOI | MR | Zbl
[27] Diffusion equation for multivalued stochastic differential equations, J. Funct. Anal., Volume 49 (1982) no. 1, pp. 73-90 | MR | Zbl | DOI
[28] Lagrangian relaxation, Computational combinatorial optimization. Optimal of probably near-optimal solutions (Lecture Notes in Computer Science), Volume 2241, 2001, pp. 112-156 | DOI | Zbl
[29] Stochastic differential equations and applications, Horwood Publishing, 2007 | Zbl | MR
[30] An Stochastic Differential Equation Perspective on Stochastic Convex Optimization, Math. Oper. Res. (2024) (Ahead of Print) | DOI
[31] NIST Handbook of Mathematical Functions (Frank W. J. Olver; Daniel w. Lozier; Ronald F. Boisvert; Charles W. Clarck, eds.), Cambridge University Press, 2010 | MR | Zbl
[32] Weak convergence of the sequence of successive approximations for nonexpansive mappings, Bull. Am. Math. Soc., Volume 73 (1967), pp. 591-597 | DOI | MR
[33] Stochastic differential equations, backward SDEs, partial differential equations, Stochastic Modelling and Applied Probability, 69, Springer, 2014 | DOI | Zbl | MR
[34] Yosida approximations for multivalued stochastic differential equations, Stochastics Stochastics Rep., Volume 52 (1994), pp. 107-120 | DOI | Zbl | MR
[35] Deterministic and Stochastic Differential Equations in Hilbert spaces involving Multivalued Maximal Monotone Operators (2014) | arXiv
[36] The Fitzpatrick function – a bridge betweem convex analysis and multivalued stochastic differential equations (2009) | arXiv
[37] A stochastic approximation method, Ann. Math. Stat., Volume 22 (1951), pp. 400-407 | DOI | Zbl | MR
[38] Convex analysis, Princeton Landmarks in Mathematics, 28, Princeton University Press, 1997 | Zbl | MR
[39] Real and complex analysis, McGraw-Hill, 1987 | Zbl | MR
[40] Abstract martingale convergence theorems, Pac. J. Math., Volume 11 (1961) no. 1, pp. 347-374 | DOI | MR | Zbl
[41] Minimization Methods for non-differentiable functions, Springer Series in Computational Mathematics, 3, Springer, 1985 | DOI | Zbl | MR
[42] A Dynamical System Perspective on Stochastic and Inertial Methods for Optimization, Ph. D. Thesis, Université de Caen Normandie (France) (2024)
[43] Continuous-time Analysis of Anchor Acceleration, NIPS ’23: Proceedings of the 37th International Conference on Neural Information Processing Systems, Curran Associates, Inc. (2023), pp. 32782-32866
[44] Convergence epigraphique et changements d’echelle en analyse variationelle et optimisation, Ph. D. Thesis, Université de Montpellier 2 (France) (1996), 160 pages
Cited by Sources: