101-150 of 10000 results (56ms)
2022-09-27 ยง
17:38 <andrew@cumin1001> START - Cookbook sre.hosts.reboot-single for host cloudvirt-wdqs1002.eqiad.wmnet [production]
17:38 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) for host cloudvirt-wdqs1003.eqiad.wmnet [production]
17:29 <jbond@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts sretest[1001-1002].eqiad.wmnet [production]
17:28 <jbond@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts sretest[1001-1002].eqiad.wmnet [production]
17:26 <andrew@cumin1001> START - Cookbook sre.hosts.reboot-single for host cloudvirt-wdqs1003.eqiad.wmnet [production]
17:19 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) for host cloudvirt-wdqs1003.eqiad.wmnet [production]
17:08 <andrew@cumin1001> START - Cookbook sre.hosts.reboot-single for host cloudvirt-wdqs1003.eqiad.wmnet [production]
14:56 <mforns@deploy1002> Finished deploy [airflow-dags/analytics@25dda27]: (no justification provided) (duration: 00m 11s) [production]
14:56 <mforns@deploy1002> Started deploy [airflow-dags/analytics@25dda27]: (no justification provided) [production]
14:38 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2141.codfw.wmnet with reason: Maintenance [production]
14:38 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2141.codfw.wmnet with reason: Maintenance [production]
14:38 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2130 (T314041)', diff saved to https://phabricator.wikimedia.org/P34958 and previous config saved to /var/cache/conftool/dbconfig/20220927-143831-ladsgroup.json [production]
14:35 <pt1979@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host logstash2036.codfw.wmnet with OS buster [production]
14:31 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1118 (T314041)', diff saved to https://phabricator.wikimedia.org/P34957 and previous config saved to /var/cache/conftool/dbconfig/20220927-143109-ladsgroup.json [production]
14:31 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1118.eqiad.wmnet with reason: Maintenance [production]
14:30 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1118.eqiad.wmnet with reason: Maintenance [production]
14:30 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1107 (T314041)', diff saved to https://phabricator.wikimedia.org/P34956 and previous config saved to /var/cache/conftool/dbconfig/20220927-143047-ladsgroup.json [production]
14:26 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host logstash2036.codfw.wmnet with OS buster [production]
14:25 <Lucas_WMDE> END lucaswerkmeister-wmde@mwmaint1002:~$ PHP=php7.4 mwscript updateCollation.php incubatorwiki --force # T315552, 710183 rows done [production]
14:23 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2130', diff saved to https://phabricator.wikimedia.org/P34955 and previous config saved to /var/cache/conftool/dbconfig/20220927-142324-ladsgroup.json [production]
14:23 <mforns@deploy1002> Finished deploy [airflow-dags/analytics@66dfa44]: (no justification provided) (duration: 00m 46s) [production]
14:22 <mforns@deploy1002> Started deploy [airflow-dags/analytics@66dfa44]: (no justification provided) [production]
14:17 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
14:15 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1107', diff saved to https://phabricator.wikimedia.org/P34954 and previous config saved to /var/cache/conftool/dbconfig/20220927-141541-ladsgroup.json [production]
14:13 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
14:13 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
14:13 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
14:11 <Lucas_WMDE> BEGIN lucaswerkmeister-wmde@mwmaint1002:~$ PHP=php7.4 mwscript updateCollation.php incubatorwiki --force # T315552 [production]
14:08 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2130', diff saved to https://phabricator.wikimedia.org/P34953 and previous config saved to /var/cache/conftool/dbconfig/20220927-140817-ladsgroup.json [production]
14:08 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
14:06 <taavi@deploy1002> Finished scap: Backport for [[gerrit:835590|Track use of Searchbox footer on Wikidata (T306933)]], [[gerrit:835591|Track use of Searchbox footer on Wikidata (T306933)]] (duration: 06m 59s) [production]
14:04 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
14:04 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
14:00 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1107', diff saved to https://phabricator.wikimedia.org/P34952 and previous config saved to /var/cache/conftool/dbconfig/20220927-140034-ladsgroup.json [production]
14:00 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
13:59 <taavi@deploy1002> taavi and migr: Backport for [[gerrit:835590|Track use of Searchbox footer on Wikidata (T306933)]], [[gerrit:835591|Track use of Searchbox footer on Wikidata (T306933)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet [production]
13:59 <taavi@deploy1002> Started scap: Backport for [[gerrit:835590|Track use of Searchbox footer on Wikidata (T306933)]], [[gerrit:835591|Track use of Searchbox footer on Wikidata (T306933)]] [production]
13:53 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2130 (T314041)', diff saved to https://phabricator.wikimedia.org/P34951 and previous config saved to /var/cache/conftool/dbconfig/20220927-135310-ladsgroup.json [production]
13:45 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1107 (T314041)', diff saved to https://phabricator.wikimedia.org/P34950 and previous config saved to /var/cache/conftool/dbconfig/20220927-134528-ladsgroup.json [production]
12:42 <klausman@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . [production]
12:36 <klausman@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . [production]
12:31 <klausman@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . [production]
12:28 <klausman@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . [production]
12:26 <klausman@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . [production]
12:23 <klausman@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . [production]
12:20 <klausman@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
12:18 <klausman@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'experimental' for release 'main' . [production]
12:15 <klausman@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . [production]
11:58 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
11:57 <jbond> upload new wmf-laptop_0.5.4 package [production]