|
2021-01-06
ยง
|
| 20:58 |
<andrew@deploy1001> |
Started deploy [horizon/deploy@965995d]: update codfw1dev deploy from train-buster branch |
[production] |
| 20:43 |
<andrew@deploy1001> |
Finished deploy [horizon/deploy@89b308c]: update codfw1dev deploy for Buster (duration: 02m 30s) |
[production] |
| 20:40 |
<andrew@deploy1001> |
Started deploy [horizon/deploy@89b308c]: update codfw1dev deploy for Buster |
[production] |
| 20:40 |
<andrew@deploy1001> |
Finished deploy [horizon/deploy@89b308c]: update codfw1dev deploy for Buster (duration: 00m 07s) |
[production] |
| 20:40 |
<andrew@deploy1001> |
Started deploy [horizon/deploy@89b308c]: update codfw1dev deploy for Buster |
[production] |
| 20:38 |
<andrew@deploy1001> |
Finished deploy [horizon/deploy@89b308c]: update codfw1dev deploy for Buster (duration: 00m 05s) |
[production] |
| 20:38 |
<andrew@deploy1001> |
Started deploy [horizon/deploy@89b308c]: update codfw1dev deploy for Buster |
[production] |
| 20:27 |
<andrew@deploy1001> |
Finished deploy [horizon/deploy@89b308c]: (no justification provided) (duration: 00m 05s) |
[production] |
| 20:27 |
<andrew@deploy1001> |
Started deploy [horizon/deploy@89b308c]: (no justification provided) |
[production] |
| 20:26 |
<andrew@deploy1001> |
Finished deploy [horizon/deploy@89b308c]: update codfw1dev deploy (duration: 00m 08s) |
[production] |
| 20:26 |
<andrew@deploy1001> |
Started deploy [horizon/deploy@89b308c]: update codfw1dev deploy |
[production] |
| 19:37 |
<eileen> |
increased batch processes on eoy_receipt - now 9000 per hour process-control config revision is 7b8319b97d |
[production] |
| 19:25 |
<dpifke> |
Morning backport window complete. |
[production] |
| 19:22 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudweb2001-dev.wikimedia.org with reason: REIMAGE |
[production] |
| 19:20 |
<dpifke@deploy1001> |
Synchronized lib: Removing unused profiler libraries (duration: 01m 03s) |
[production] |
| 19:19 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cloudweb2001-dev.wikimedia.org with reason: REIMAGE |
[production] |
| 19:18 |
<dpifke@deploy1001> |
Synchronized wmf-config/PhpAutoPrepend.php: Removing unused profiler code (duration: 01m 04s) |
[production] |
| 19:17 |
<dpifke@deploy1001> |
Synchronized wmf-config/profiler.php: Removing unused profiler code (duration: 01m 08s) |
[production] |
| 19:13 |
<eileen> |
process-control config revision is 2909e8b9ed |
[production] |
| 19:11 |
<eileen> |
civicrm revision changed from 1d5f6365ba to 95fefcb421, config revision is 9bc3d67b02 |
[production] |
| 18:14 |
<mutante> |
creating uz.wikimedia.org - Uzbek language User Group - https://meta.wikimedia.org/wiki/Affiliations_Committee/Resolutions/Wikimedians_of_the_Uzbek_language_User_Group |
[production] |
| 17:53 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on labweb1002.wikimedia.org with reason: REIMAGE |
[production] |
| 17:51 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on labweb1002.wikimedia.org with reason: REIMAGE |
[production] |
| 17:26 |
<andrewbogott> |
depooling labweb1002 for rebuild |
[production] |
| 17:00 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc2026.codfw.wmnet with reason: REIMAGE |
[production] |
| 16:58 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1026.eqiad.wmnet with reason: REIMAGE |
[production] |
| 16:58 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on mc2026.codfw.wmnet with reason: REIMAGE |
[production] |
| 16:56 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on mc1026.eqiad.wmnet with reason: REIMAGE |
[production] |
| 16:42 |
<razzi@cumin1001> |
END (PASS) - Cookbook sre.hadoop.roll-restart-masters (exit_code=0) |
[production] |
| 16:32 |
<jayme@deploy1001> |
helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
| 16:31 |
<jayme@deploy1001> |
helmfile [staging-codfw] START helmfile.d/admin 'sync'. |
[production] |
| 16:15 |
<jayme@deploy1001> |
helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
| 16:15 |
<jayme@deploy1001> |
helmfile [staging-codfw] START helmfile.d/admin 'sync'. |
[production] |
| 16:02 |
<razzi@cumin1001> |
START - Cookbook sre.hadoop.roll-restart-masters |
[production] |
| 16:01 |
<moritzm> |
installing cups security updates on buster (client-side tools/libs) |
[production] |
| 15:54 |
<moritzm> |
installing openexr security updates |
[production] |
| 15:28 |
<jayme> |
imported calico 3.17.1-1 to component/calico-future stretch-wikimedia |
[production] |
| 15:20 |
<moritzm> |
restarting FPM/Apache on mw canaries to pick up p11-kit update |
[production] |
| 15:04 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 15:02 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 15:01 |
<moritzm> |
installing p11-kit security updates on stretch |
[production] |
| 14:58 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 14:55 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 13:03 |
<moritzm> |
installing tcpdump security updates |
[production] |
| 12:11 |
<awight@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:654005|Create rollbacker group on mrwiki (T270864)]] (duration: 01m 21s) |
[production] |
| 11:33 |
<moritzm> |
installing ruby2.5 security updates |
[production] |
| 11:18 |
<moritzm> |
installing libjpeg-turbo security updates on buster |
[production] |
| 11:14 |
<moritzm> |
remove cloudceph2002-dev.wikimedia.org and cloudceph2003-dev.wikimedia.org from debmonitor (got reinstalled as .wmnet) |
[production] |
| 10:40 |
<jmm@cumin2001> |
dbctl commit (dc=all): 'Depool db2140', diff saved to https://phabricator.wikimedia.org/P13658 and previous config saved to /var/cache/conftool/dbconfig/20210106-104029-jmm.json |
[production] |
| 10:38 |
<moritzm> |
depooling db2140 T271084 |
[production] |