All CPUs busy when no user processes

Support and Help
, ,
1

Ubuntu Version

Ubuntu 24.04 LTS

Desktop Environment (if applicable)

Other

Problem Description

All ssh connection will drop and cpu goes up to 100 percent usage

Relevant System Information

CPU settings 64GB
Memory 64GB

What I’ve Tried

Upscale the cpu from 16 GB to 64GB. still cpu is 100% usage

CPU output

ps -eo pid,ppid,user,%cpu,%mem,cmd --sort=-%cpu | grep sshd
3606058 3525 root 133 0.0 sshd: [accepted]
3605981 3525 root 94.2 0.0 sshd: [accepted]
3604782 3525 root 84.4 0.0 sshd: [accepted]
3605682 3525 root 83.7 0.0 sshd: [accepted]
3604568 3525 root 82.0 0.0 sshd: [accepted]
3605883 3525 root 80.4 0.0 sshd: [accepted]
3605264 3525 root 77.8 0.0 sshd: [accepted]
3603853 3525 root 74.9 0.0 sshd: [accepted]
3605287 3525 root 72.7 0.0 sshd: [accepted]
3603105 3525 root 72.4 0.0 sshd: root [priv]
3605071 3525 root 71.5 0.0 sshd: [accepted]
3602536 3525 root 70.0 0.0 sshd: [accepted]
3605182 3525 root 67.2 0.0 sshd: [accepted]
3604077 3525 root 64.1 0.0 sshd: [accepted]
3603868 3525 root 62.6 0.0 sshd: [accepted]
3604978 3525 root 61.9 0.0 sshd: [accepted]
3604175 3525 root 60.9 0.0 sshd: [accepted]
3604644 3525 root 59.7 0.0 sshd: [accepted]
3605982 3525 root 59.6 0.0 sshd: [accepted]
3602661 3525 root 59.4 0.0 sshd: [accepted]
3603372 3525 root 59.2 0.0 sshd: [accepted]
3604853 3525 root 59.0 0.0 sshd: [accepted]
3602911 3525 root 58.9 0.0 sshd: [accepted]
3604781 3525 root 57.6 0.0 sshd: [accepted]
3603863 3525 root 57.1 0.0 sshd: [accepted]
3602004 3525 root 56.2 0.0 sshd: [accepted]
3597001 3525 root 55.7 0.0 sshd: [accepted]
3603906 3525 root 55.6 0.0 sshd: [accepted]
3600850 3525 root 55.0 0.0 sshd: [accepted]
3597640 3525 root 54.4 0.0 sshd: [accepted]
3602778 3525 root 54.1 0.0 sshd: [accepted]
3601536 3525 root 52.9 0.0 sshd: [accepted]
3602339 3525 root 52.7 0.0 sshd: [accepted]
3606043 3525 root 52.5 0.0 sshd: [accepted]
3602652 3525 root 52.5 0.0 sshd: [accepted]
3601906 3525 root 52.4 0.0 sshd: [accepted]
3604124 3525 root 52.0 0.0 sshd: [accepted]
3597645 3525 root 50.7 0.0 sshd: [accepted]
3595473 3525 root 50.7 0.0 sshd: [accepted]
3595259 3525 root 50.5 0.0 sshd: [accepted]
3604104 3525 root 50.5 0.0 sshd: [accepted]
3602003 3525 root 50.5 0.0 sshd: [accepted]
3604783 3525 root 49.8 0.0 sshd: [accepted]
3603670 3525 root 49.7 0.0 sshd: [accepted]
3595254 3525 root 49.6 0.0 sshd: [accepted]
3595258 3525 root 49.5 0.0 sshd: [accepted]
3601357 3525 root 49.3 0.0 sshd: [accepted]
3602222 3525 root 49.1 0.0 sshd: [accepted]
3603673 3525 root 48.8 0.0 sshd: [accepted]
3604499 3525 root 48.5 0.0 sshd: [accepted]
3595458 3525 root 48.4 0.0 sshd: [accepted]
3601932 3525 root 48.1 0.0 sshd: [accepted]
3602644 3525 root 47.8 0.0 sshd: [accepted]
3600762 3525 root 47.8 0.0 sshd: [accepted]
3601556 3525 root 47.7 0.0 sshd: [accepted]
3603445 3525 root 47.3 0.0 sshd: [accepted]
3601208 3525 root 47.2 0.0 sshd: [accepted]
3597687 3525 root 47.1 0.0 sshd: [accepted]
3597130 3525 root 47.1 0.0 sshd: [accepted]
3596992 3525 root 46.6 0.0 sshd: [accepted]
3603256 3525 root 46.6 0.0 sshd: [accepted]
3597663 3525 root 46.4 0.0 sshd: [accepted]
3601923 3525 root 46.4 0.0 sshd: [accepted]
3601909 3525 root 46.2 0.0 sshd: [accepted]
3602002 3525 root 46.1 0.0 sshd: [accepted]
3602114 3525 root 45.8 0.0 sshd: [accepted]
3604214 3525 root 45.5 0.0 sshd: [accepted]
3597195 3525 root 45.4 0.0 sshd: [accepted]
3602088 3525 root 44.7 0.0 sshd: [accepted]
3596941 3525 root 44.6 0.0 sshd: [accepted]
3595280 3525 root 44.5 0.0 sshd: [accepted]
3602780 3525 root 44.0 0.0 sshd: [accepted]
3605652 3525 root 42.8 0.0 sshd: [accepted]
3601189 3525 root 42.8 0.0 sshd: [accepted]
3601423 3525 root 42.3 0.0 sshd: [accepted]
3602155 3525 root 42.3 0.0 sshd: [accepted]
3601618 3525 root 42.1 0.0 sshd: [accepted]
3604335 3525 root 41.8 0.0 sshd: [accepted]
3595282 3525 root 41.8 0.0 sshd: [accepted]
3603113 3525 root 41.2 0.0 sshd: [accepted]
3603597 3525 root 40.6 0.0 sshd: [accepted]
3602551 3525 root 39.4 0.0 sshd: [accepted]
3602005 3525 root 39.2 0.0 sshd: [accepted]
3598900 3525 root 39.1 0.0 sshd: [accepted]
3597762 3525 root 38.8 0.0 sshd: [accepted]
3602338 3525 root 38.7 0.0 sshd: [accepted]
3600697 3525 root 38.6 0.0 sshd: [accepted]
3603861 3525 root 38.2 0.0 sshd: [accepted]
3604790 3525 root 38.2 0.0 sshd: [accepted]
3602788 3525 root 37.4 0.0 sshd: [accepted]
3602006 3525 root 37.4 0.0 sshd: [accepted]
3604521 3525 root 36.7 0.0 sshd: [accepted]
3605718 3525 root 36.7 0.0 sshd: [accepted]
3597578 3525 root 36.7 0.0 sshd: [accepted]
3604212 3525 root 35.5 0.0 sshd: [accepted]
3603173 3525 root 35.2 0.0 sshd: [accepted]
3605651 3525 root 34.8 0.0 sshd: [accepted]
3595661 3525 root 31.5 0.0 sshd: [accepted]
3605196 3525 root 31.4 0.0 sshd: [accepted]
3594590 3525 root 31.1 0.0 sshd: [accepted]
3601142 3525 root 30.9 0.0 sshd: [accepted]
3604053 3525 root 30.3 0.0 sshd: [accepted]
3601209 3525 root 29.8 0.0 sshd: [accepted]
3596747 3525 root 29.6 0.0 sshd: [accepted]
3603098 3525 root 28.9 0.0 sshd: [accepted]
3599919 3525 root 28.7 0.0 sshd: [accepted]
3605164 3525 root 28.5 0.0 sshd: [accepted]
3600562 3525 root 28.5 0.0 sshd: [accepted]
3605411 3525 root 28.3 0.0 sshd: [accepted]
3600063 3525 root 28.0 0.0 sshd: [accepted]
3600487 3525 root 27.3 0.0 sshd: [accepted]
3600050 3525 root 26.6 0.0 sshd: [accepted]
3603303 3525 root 26.6 0.0 sshd: [accepted]
3599742 3525 root 26.5 0.0 sshd: [accepted]
3605786 3525 root 25.6 0.0 sshd: [accepted]
3599911 3525 root 24.8 0.0 sshd: [accepted]
3599789 3525 root 24.7 0.0 sshd: [accepted]
3600766 3525 root 24.1 0.0 sshd: [accepted]
3600051 3525 root 22.5 0.0 sshd: [accepted]
3598681 3525 root 22.3 0.0 sshd: [accepted]
3600199 3525 root 22.2 0.0 sshd: [accepted]
3600953 3525 root 22.1 0.0 sshd: [accepted]
3600268 3525 root 21.8 0.0 sshd: [accepted]
3600769 3525 root 21.0 0.0 sshd: [accepted]
3600048 3525 root 20.5 0.0 sshd: [accepted]
3600975 3525 root 19.9 0.0 sshd: [accepted]
3600257 3525 root 18.7 0.0 sshd: [accepted]
3600964 3525 root 17.0 0.0 sshd: [accepted]
3600931 3525 root 12.7 0.0 sshd: [accepted]
3600882 3525 root 11.6 0.0 sshd: [accepted]
3606056 3525 root 9.7 0.0 sshd: [accepted]
3602925 3525 root 9.7 0.0 sshd: [accepted]
3603175 3525 root 9.1 0.0 sshd: [accepted]
3606048 3525 root 4.6 0.0 sshd: [accepted]
3606054 3525 root 4.5 0.0 sshd: [accepted]
3606051 3525 root 4.2 0.0 sshd: [accepted]
3606050 3525 root 4.1 0.0 sshd: [accepted]
3606045 3525 root 3.8 0.0 sshd: [accepted]
3426238 3525 root 1.8 0.0 sshd: firstgencorp1 [priv]
3605977 3525 root 1.8 0.0 sshd: [accepted]
3605974 3525 root 1.7 0.0 sshd: [accepted]
3605863 3525 root 1.5 0.0 sshd: [accepted]
3605850 3525 root 1.4 0.0 sshd: [accepted]
3605793 3525 root 1.0 0.0 sshd: [accepted]
3605789 3525 root 1.0 0.0 sshd: [accepted]
3605791 3525 root 0.8 0.0 sshd: [accepted]
3525 1 root 0.7 0.0 sshd: /usr/sbin/sshd -D [listener] 167 of 200-1000 startups
3605783 3525 root 0.7 0.0 sshd: [accepted]
3605478 3525 root 0.6 0.0 sshd: [accepted]
3605829 3525 root 0.6 0.0 sshd: [accepted]
3605416 3525 root 0.6 0.0 sshd: [accepted]
3605461 3525 root 0.5 0.0 sshd: [accepted]
3605723 3525 root 0.5 0.0 sshd: [accepted]
3605414 3525 root 0.5 0.0 sshd: [accepted]
3604864 3525 root 0.4 0.0 sshd: [accepted]
3605292 3525 root 0.4 0.0 sshd: [accepted]
3605288 3525 root 0.3 0.0 sshd: [accepted]
3605223 3525 root 0.3 0.0 sshd: [accepted]
3605221 3525 root 0.3 0.0 sshd: [accepted]
3605220 3525 root 0.3 0.0 sshd: [accepted]
3604754 3525 root 0.3 0.0 sshd: [accepted]
3605721 3525 root 0.3 0.0 sshd: [accepted]
3605719 3525 root 0.3 0.0 sshd: [accepted]
3604860 3525 root 0.2 0.0 sshd: [accepted]
3605296 3525 root 0.2 0.0 sshd: [accepted]
3605293 3525 root 0.2 0.0 sshd: [accepted]
3604858 3525 root 0.1 0.0 sshd: [accepted]
3604855 3525 root 0.1 0.0 sshd: [accepted]
3604792 3525 root 0.1 0.0 sshd: [accepted]
3055781 3525 root 0.0 0.0 sshd: p_pam_admin [priv]
135361 3525 root 0.0 0.0 sshd: p_grabpay_sftpclnt [priv]
8587 3525 root 0.0 0.0 sshd: p_sapsf_sftp [priv]
3058058 3055781 p_pam_a+ 0.0 0.0 sshd: p_pam_admin@pts/1
3431337 3426238 firstge+ 0.0 0.0 sshd: firstgencorp1
9428 8587 p_sapsf+ 0.0 0.0 sshd: p_sapsf_sftp@notty
3606062 3525 root 0.0 0.0 sshd: [accepted]
9437 9428 p_sapsf+ 0.0 0.0 sshd: p_sapsf_sftp@internal-sftp
136361 135361 p_grabp+ 0.0 0.0 sshd: p_grabpay_sftpclnt@notty
136362 136361 p_grabp+ 0.0 0.0 sshd: p_grabpay_sftpclnt@internal-sftp
3595264 3595258 sshd 0.0 0.0 sshd: [net]
3595265 3595254 sshd 0.0 0.0 sshd: [net]
3595277 3595259 sshd 0.0 0.0 sshd: [net]
3595284 3595280 sshd 0.0 0.0 sshd: [net]
3595285 3595282 sshd 0.0 0.0 sshd: [net]
3595462 3595458 sshd 0.0 0.0 sshd: [net]
3595477 3595473 sshd 0.0 0.0 sshd: [net]
3596051 3594590 sshd 0.0 0.0 sshd: [net]
3596984 3596941 sshd 0.0 0.0 sshd: [net]
3596999 3596992 sshd 0.0 0.0 sshd: [net]
3597006 3597001 sshd 0.0 0.0 sshd: [net]
3597134 3597130 sshd 0.0 0.0 sshd: [net]
3597201 3597195 sshd 0.0 0.0 sshd: [net]
3597323 3595661 sshd 0.0 0.0 sshd: [net]
3597606 3597578 sshd 0.0 0.0 sshd: [net]
3597643 3597640 sshd 0.0 0.0 sshd: [net]
3597651 3597645 sshd 0.0 0.0 sshd: [net]
3597697 3597663 sshd 0.0 0.0 sshd: [net]
3597714 3597687 sshd 0.0 0.0 sshd: [net]
3597781 3597762 sshd 0.0 0.0 sshd: [net]
3598236 3596747 sshd 0.0 0.0 sshd: [net]
3598902 3598900 sshd 0.0 0.0 sshd: [net]
3599922 3599919 sshd 0.0 0.0 sshd: [net]
3600320 3598681 sshd 0.0 0.0 sshd: [net]
3600699 3600697 sshd 0.0 0.0 sshd: [net]
3600765 3600762 sshd 0.0 0.0 sshd: [net]
3600898 3600850 sshd 0.0 0.0 sshd: [net]
3600966 3599911 sshd 0.0 0.0 sshd: [net]
3601041 3599742 sshd 0.0 0.0 sshd: [net]
3601042 3599789 sshd 0.0 0.0 sshd: [net]
3601044 3600063 sshd 0.0 0.0 sshd: [net]
3601045 3600051 sshd 0.0 0.0 sshd: [net]
3601046 3600048 sshd 0.0 0.0 sshd: [net]
3601047 3600050 sshd 0.0 0.0 sshd: [net]
3601061 3600199 sshd 0.0 0.0 sshd: [net]
3601065 3600257 sshd 0.0 0.0 sshd: [net]
3601147 3601142 sshd 0.0 0.0 sshd: [net]
3601206 3601189 sshd 0.0 0.0 sshd: [net]
3601213 3601208 sshd 0.0 0.0 sshd: [net]
3601214 3601209 sshd 0.0 0.0 sshd: [net]
3601292 3600268 sshd 0.0 0.0 sshd: [net]
3601379 3601357 sshd 0.0 0.0 sshd: [net]
3601426 3601423 sshd 0.0 0.0 sshd: [net]
3601550 3601536 sshd 0.0 0.0 sshd: [net]
3601555 3600487 sshd 0.0 0.0 sshd: [net]
3601558 3601556 sshd 0.0 0.0 sshd: [net]
3601621 3601618 sshd 0.0 0.0 sshd: [net]
3601928 3601906 sshd 0.0 0.0 sshd: [net]
3601929 3601909 sshd 0.0 0.0 sshd: [net]
3601930 3601923 sshd 0.0 0.0 sshd: [net]
3601934 3601932 sshd 0.0 0.0 sshd: [net]
3601972 3600562 sshd 0.0 0.0 sshd: [net]
3602012 3602002 sshd 0.0 0.0 sshd: [net]
3602013 3602004 sshd 0.0 0.0 sshd: [net]
3602015 3602005 sshd 0.0 0.0 sshd: [net]
3602016 3602003 sshd 0.0 0.0 sshd: [net]
3602017 3602006 sshd 0.0 0.0 sshd: [net]
3602025 3600766 sshd 0.0 0.0 sshd: [net]
3602042 3600769 sshd 0.0 0.0 sshd: [net]
3602090 3602088 sshd 0.0 0.0 sshd: [net]
3602135 3600931 sshd 0.0 0.0 sshd: [net]
3602150 3602114 sshd 0.0 0.0 sshd: [net]
3602159 3602155 sshd 0.0 0.0 sshd: [net]
3602316 3602222 sshd 0.0 0.0 sshd: [net]
3602342 3602339 sshd 0.0 0.0 sshd: [net]
3602343 3602338 sshd 0.0 0.0 sshd: [net]
3602569 3602536 sshd 0.0 0.0 sshd: [net]
3602587 3602551 sshd 0.0 0.0 sshd: [net]
3602657 3602644 sshd 0.0 0.0 sshd: [net]
3602666 3602652 sshd 0.0 0.0 sshd: [net]
3602689 3602661 sshd 0.0 0.0 sshd: [net]
3602781 3602778 sshd 0.0 0.0 sshd: [net]
3602783 3600975 sshd 0.0 0.0 sshd: [net]
3602784 3600964 sshd 0.0 0.0 sshd: [net]
3602785 3600882 sshd 0.0 0.0 sshd: [net]
3602786 3600953 sshd 0.0 0.0 sshd: [net]
3602787 3602780 sshd 0.0 0.0 sshd: [net]
3602793 3602788 sshd 0.0 0.0 sshd: [net]
3602917 3602911 sshd 0.0 0.0 sshd: [net]
3603102 3603098 sshd 0.0 0.0 sshd: [net]
3603140 3603113 sshd 0.0 0.0 sshd: [net]
3603177 3603173 sshd 0.0 0.0 sshd: [net]
3603291 3603256 sshd 0.0 0.0 sshd: [net]
3603343 3603303 sshd 0.0 0.0 sshd: [net]
3603404 3603372 sshd 0.0 0.0 sshd: [net]
3603473 3603445 sshd 0.0 0.0 sshd: [net]
3603642 3603597 sshd 0.0 0.0 sshd: [net]
3603786 3603673 sshd 0.0 0.0 sshd: [net]
3603818 3603670 sshd 0.0 0.0 sshd: [net]
3603888 3603853 sshd 0.0 0.0 sshd: [net]
3603897 3603868 sshd 0.0 0.0 sshd: [net]
3603901 3603861 sshd 0.0 0.0 sshd: [net]
3603904 3603863 sshd 0.0 0.0 sshd: [net]
3603908 3603906 sshd 0.0 0.0 sshd: [net]
3604085 3604053 sshd 0.0 0.0 sshd: [net]
3604096 3604077 sshd 0.0 0.0 sshd: [net]
3604141 3604104 sshd 0.0 0.0 sshd: [net]
3604150 3604124 sshd 0.0 0.0 sshd: [net]
3604207 3604175 sshd 0.0 0.0 sshd: [net]
3604215 3604212 sshd 0.0 0.0 sshd: [net]
3604217 3604214 sshd 0.0 0.0 sshd: [net]
3604337 3604335 sshd 0.0 0.0 sshd: [net]
3604520 3604499 sshd 0.0 0.0 sshd: [net]
3604536 3604521 sshd 0.0 0.0 sshd: [net]
3604627 3604568 sshd 0.0 0.0 sshd: [net]
3604646 3604644 sshd 0.0 0.0 sshd: [net]
3604787 3604782 sshd 0.0 0.0 sshd: [net]
3604788 3604781 sshd 0.0 0.0 sshd: [net]
3604789 3604783 sshd 0.0 0.0 sshd: [net]
3604793 3604790 sshd 0.0 0.0 sshd: [net]
3604857 3604853 sshd 0.0 0.0 sshd: [net]
3605027 3604978 sshd 0.0 0.0 sshd: [net]
3605150 3605071 sshd 0.0 0.0 sshd: [net]
3605202 3605164 sshd 0.0 0.0 sshd: [net]
3605209 3605182 sshd 0.0 0.0 sshd: [net]
3605217 3605196 sshd 0.0 0.0 sshd: [net]
3605284 3605264 sshd 0.0 0.0 sshd: [net]
3605291 3605287 sshd 0.0 0.0 sshd: [net]
3605413 3605411 sshd 0.0 0.0 sshd: [net]
3605587 3602925 sshd 0.0 0.0 sshd: [net]
3605655 3605651 sshd 0.0 0.0 sshd: [net]
3605656 3605652 sshd 0.0 0.0 sshd: [net]
3605715 3605682 sshd 0.0 0.0 sshd: [net]
3605724 3603175 sshd 0.0 0.0 sshd: [net]
3605727 3605718 sshd 0.0 0.0 sshd: [net]
3605788 3605786 sshd 0.0 0.0 sshd: [net]
3605938 3605883 sshd 0.0 0.0 sshd: [net]
3605985 3605982 sshd 0.0 0.0 sshd: [net]
3605986 3605981 sshd 0.0 0.0 sshd: [net]
3606046 3606043 sshd 0.0 0.0 sshd: [net]
3606060 3060187 root 0.0 0.0 grep --color=auto sshd

================================================================

For CONTEXT:

We recently performed an SFTP migration from Ubuntu 18 to Ubuntu 24. During cutover, the Ubuntu 24 server inherited the same hostname, aliases, public IP, and private IP as the existing production server.

Under what we believe to be the same workload, Ubuntu 24 experienced:

  • Sustained high CPU utilization

  • Large numbers of sshd: [accepted] and sshd: root [priv] processes consuming CPU

  • SSH/SFTP login delays during session establishment

  • MaxStartups throttling

The production Ubuntu 18 server currently handles similar or even higher concurrent SSH session counts without exhibiting the same symptoms.

One notable observation is that an internal synchronization server (rsync over SSH) generated significantly more SSH authentications during the Ubuntu 24 cutover than what we currently observe against Ubuntu 18, despite being the same source system running the same synchronization workload.

Has anyone encountered similar behavior after moving from Ubuntu 18/OpenSSH 7.x to Ubuntu 24/OpenSSH 9.x, particularly around authentication rates, sshd:[accepted] CPU usage, or pre-authentication session buildup?

2 Likes
2

Welcome to the community @alvinsambas!
Try to identify the processes using the most CPU:
ps -eo pid,ppid,user,%cpu,%mem,cmd --sort=-%cpu
Thank you for the feedback.

1 Like
3

You are being brute-forced.

Those sshd: [accepted] processes are doing key exchange crypto for incoming connections. Dozens of bots hammering port 22 at once will max out any CPU on Diffie-Hellman / Curve25519 math alone.

Check who is hitting you:

ss -tnp sport = :22 | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr | head

Quick fix right now:

sudo ufw limit 22/tcp

Then open /etc/ssh/sshd_config , set MaxStartups 10:30:60 , and restart ssh.

After that, install fail2ban or firewall port 22 to known IPs.
If you expose ports to the outside world, that is an absolute must-have.

4 Likes
4

On top of the suggestions from @steelf.
@alvinsambas there were cases in the past of this pattern in 22.04 not linked to bruteforce.
You could also try to run ps -ef | grep -w ssh | grep -v grep | grep -v pts

5

Another ps will not show anything new, it just lists the same child workers.

The ss command from earlier gives you the remote IP immediately.
Whether it is an external botnet or a broken internal healthcheck hitting port 22, the IP will show up right at the top of the list.

2 Likes
6

hi sir thank you for your reply. this is my

kexalgorithms curve25519-sha256@libssh.org,ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group-exchange-sha256
casignaturealgorithms ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256

is there any recommendation aside from ufw setup?

7

Take a look at the fail2ban tool …

2 Likes
8

Changing KexAlgorithms will not change anything. Every key exchange burns CPU when bots hammer the port, because crypto happens before authentication.

Drop the flood before it reaches sshd. Set MaxStartups 10:30:60 in /etc/ssh/sshd_config, install fail2ban as @ogra pointed out, or move SSH off port 22.

2 Likes
9

Hi sir

For CONTEXT:

We recently performed an SFTP migration from Ubuntu 18 to Ubuntu 24. During cutover, the Ubuntu 24 server inherited the same hostname, aliases, public IP, and private IP as the existing production server.

Under what we believe to be the same workload, Ubuntu 24 experienced:

  • Sustained high CPU utilization

  • Large numbers of sshd: [accepted] and sshd: root [priv] processes consuming CPU

  • SSH/SFTP login delays during session establishment

  • MaxStartups throttling

The production Ubuntu 18 server currently handles similar or even higher concurrent SSH session counts without exhibiting the same symptoms.

One notable observation is that an internal synchronization server (rsync over SSH) generated significantly more SSH authentications during the Ubuntu 24 cutover than what we currently observe against Ubuntu 18, despite being the same source system running the same synchronization workload.

Has anyone encountered similar behavior after moving from Ubuntu 18/OpenSSH 7.x to Ubuntu 24/OpenSSH 9.x, particularly around authentication rates, sshd:[accepted] CPU usage, or pre-authentication session buildup?

10

That context changes everything.
If rsync is suddenly generating far more authentications, connections are failing and retrying in a tight loop.

OpenSSH 9 rejects old RSA SHA-1 keys by default. If your sync scripts or clients use ssh-rsa, authentication fails immediately. Test allowing it in /etc/ssh/sshd_config with:

PubkeyAcceptedAlgorithms +ssh-rsa

Also verify that you copied the original /etc/ssh/ssh_host_* keys from the Ubuntu 18 server.
If host keys changed, scripts with strict host checking fail and retry constantly

Once failed attempts hit the default MaxStartups 10:30:100, sshd starts dropping incoming handshakes, causing an avalanche of retries. Bump it to MaxStartups 100:30:500 to stop dropping legitimate connections while you check what the rsync client logs actually say.

1 Like
11

is there impact on scripting process? this my ubuntu 24 setup regarding cpu

lscpu
Architecture:             x86_64
CPU op-mode(s):         32-bit, 64-bit
Address sizes:          45 bits physical, 48 bits virtual
Byte Order:             Little Endian
CPU(s):                   64
On-line CPU(s) list:    0-63
Vendor ID:                GenuineIntel
BIOS Vendor ID:         GenuineIntel
Model name:             Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
BIOS Model name:      Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz  CPU @ 2.1GHz
BIOS CPU family:      2
CPU family:           6
Model:                85
Thread(s) per core:   1
Core(s) per socket:   1
Socket(s):            64

while this is my ubuntu 18

lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              64
On-line CPU(s) list: 0-63
Thread(s) per core:  1
Core(s) per socket:  4
Socket(s):           16
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               85
Model name:          Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
12

No, CPU topology is not what makes sshd spin at 100%.

While 64 virtual sockets with 1 core each is a broken hypervisor configuration (it should be 2 sockets x 32 cores or 4 x 16 to avoid NUMA overhead), it does not generate connections out of thin air.

The CPU is burning on cryptographic handshakes because your rsync jobs are failing authentication and retrying in a tight loop.

Check the actual authentication failure reason

sudo journalctl -u ssh -n 50 --no-pager

Look for lines mentioning userauth_pubkey, ssh-rsa, or Connection closed by authenticating user.
What does it report?

2 Likes
13

this i got from my ubuntu 24 machine and flooding.

Unable to negotiate with 85.17.68.225 port 58001: no matching key exchange method found. Their offer: diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,rsa2048-sha256,rsa1024-sha1 [preauth]

1 Like
14

That IP is your sync client.

It only speaks legacy SHA-1, which OpenSSH 9 dropped. It fails and retries non-stop, killing your CPU.

Add this to /etc/ssh/sshd_config:

KexAlgorithms +diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha1

Run sudo systemctl reload ssh and watch the CPU drop

2 Likes
15

Have we checked this, please? To increase the context around the root cause analysis?

sshd -V 2>&1

16

The OP mentioned Ubuntu 24 earlier, so it is OpenSSH 9.6. :detective:
And the preauth log already caught the missing SHA-1 kex. :wink:

1 Like