451-500 of 10000 results (45ms)
2021-12-15 §
12:43 <dcaro@cumin1001> START - Cookbook sre.hosts.downtime for 30 days, 0:00:00 on cloudvirt-wdqs1001.eqiad.wmnet with reason: T297454 [production]
12:43 <dcaro@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on cloudvirt-wdqs1001.eqiad.wmnet with reason: T297454 [production]
12:43 <dcaro@cumin1001> START - Cookbook sre.hosts.downtime for 4:00:00 on cloudvirt-wdqs1001.eqiad.wmnet with reason: T297454 [production]
12:40 <Lucas_WMDE> UTC morning backport+config window done [production]
12:36 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:746928|Enable Lexeme Lua access on first four wikis (T294159)]] happy holidays :) (duration: 01m 06s) [production]
12:33 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
12:32 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
12:23 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
12:22 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
12:17 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2150.codfw.wmnet with reason: Maintenance [production]
12:16 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2150.codfw.wmnet with reason: Maintenance [production]
12:13 <oblivian@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
12:11 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:745220|Remove redundant project namespace aliases (T296643)]] (no-op) (duration: 01m 07s) [production]
12:10 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
12:08 <moritzm> added ganeti2025 to codfw ganeti cluster T282603 [production]
11:48 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2150.codfw.wmnet with reason: Maintenance [production]
11:48 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2150.codfw.wmnet with reason: Maintenance [production]
11:39 <_joe_> repooling mw1414 T297667 [production]
11:36 <_joe_> upgrading php7.2 on mw1414, T297667 [production]
11:35 <_joe_> uploading php 7.2 7.2.34-18+0~20210223.60+debian10~1.gbpb21322+wmf4 to buster T297667 [production]
11:17 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2018.codfw.wmnet [production]
11:11 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti2018.codfw.wmnet [production]
10:27 <elukey@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
10:27 <elukey@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
10:00 <elukey@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . [production]
09:57 <elukey@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
09:53 <elukey@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality' for release 'main' . [production]
09:43 <elukey@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
09:42 <elukey@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
09:28 <vgutierrez> pool cp4025 - T271421 [production]
09:17 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ganeti2018.codfw.wmnet with OS buster [production]
08:44 <jmm@cumin2002> START - Cookbook sre.hosts.reimage for host ganeti2018.codfw.wmnet with OS buster [production]
07:04 <marostegui> Enable full_crc32 on db2094 (s1, s3, s5 and s8) T287244 [production]
05:48 <eileen> revision 1ede5365 -> d4cea6a9 civicrm [production]
05:07 <eileen> revision d0ac9184 -> 1ede5365 civicrm [production]
00:59 <dancy@deploy1002> Finished scap: testing (duration: 03m 38s) [production]
00:55 <dancy@deploy1002> Started scap: testing [production]
00:52 <dancy@deploy1002> Synchronized /: testing (duration: 00m 37s) [production]
00:47 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
00:41 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
00:31 <catrope@deploy1002> Synchronized php-1.38.0-wmf.13/extensions/MediaSearch/resources/store/index.js: Backport: [[gerrit:747081|Remove multiple instance of VUEX initialization (T297690)]] (duration: 01m 04s) [production]
00:29 <catrope@deploy1002> Synchronized php-1.38.0-wmf.13/extensions/MediaSearch/resources/components/SearchResults.vue: Backport: [[gerrit:747078|Don't attempt to scroll to a non-existing result]] (duration: 01m 05s) [production]
00:28 <catrope@deploy1002> Synchronized php-1.38.0-wmf.13/includes/: Backport: [[gerrit:747079|Revert "Replace deprecated methods IContextSource::getWikiPage && IContextSource::canUseWikiPage" (T297744)]] (duration: 01m 12s) [production]
00:26 <catrope@deploy1002> Synchronized php-1.38.0-wmf.12/includes/: Backport: [[gerrit:747080|Revert "Replace deprecated methods IContextSource::getWikiPage && IContextSource::canUseWikiPage" (T297744)]] (duration: 01m 11s) [production]
00:01 <bblack> lvs1015: start pybal, back to normal [production]
2021-12-14 §
23:49 <bblack> lvs1015 (internal services) - disabling pybal, will fail over traffic to lvs1020 (to test lvs1020 sanity) [production]
23:44 <bblack> lvs1013 (text) restart pybal, back to normal [production]
23:28 <bblack> lvs1013 (text) - disabling pybal, will fail over traffic to lvs1020 (to test lvs1020 sanity) [production]
23:26 <bblack> lvs1014 (upload) restart pybal, back to normal [production]
23:15 <bblack> lvs1014 (upload) - disabling pybal, will over traffic to lvs1020 (to test lvs1020 sanity) [production]