4501-4550 of 10000 results (50ms)
2021-12-16 ยง
16:49 <cwhite> pruned jndilookup.class from log4j-core on logstash 5 instances T297468 [production]
16:48 <mmandere@cumin1001> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host bast6001.wikimedia.org [production]
16:48 <mmandere@cumin1001> START - Cookbook sre.ganeti.makevm for new host bast6001.wikimedia.org [production]
16:48 <cmjohnson@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ganeti1026.eqiad.wmnet with OS buster [production]
16:47 <cmjohnson@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ganeti1025.eqiad.wmnet with OS buster [production]
16:40 <cmjohnson@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ganeti1028.eqiad.wmnet with OS buster [production]
16:39 <cmjohnson@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ganeti1027.eqiad.wmnet with OS buster [production]
16:34 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
16:33 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
16:32 <ladsgroup@deploy1002> Synchronized php-1.38.0-wmf.12/extensions/intersection: Backport: [[gerrit:747698|Set a maximum allowed time for db queries (T297708)]] (duration: 01m 06s) [production]
16:30 <ladsgroup@deploy1002> Synchronized php-1.38.0-wmf.13/extensions/intersection: Backport: [[gerrit:747697|Set a maximum allowed time for db queries (T297708)]] (duration: 01m 05s) [production]
16:28 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kubernetes1022.eqiad.wmnet with OS bullseye [production]
16:27 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
16:26 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host prometheus1006.eqiad.wmnet with OS bullseye [production]
16:26 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
16:25 <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.38.0-wmf.13/extensions/MediaSearch/: Backport: [[gerrit:747690|Filter out non-string keys/values from query string before using (T297828)]] (duration: 01m 06s) [production]
16:21 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host prometheus1005.eqiad.wmnet with OS bullseye [production]
16:09 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host ganeti1028.eqiad.wmnet with OS buster [production]
16:09 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host ganeti1027.eqiad.wmnet with OS buster [production]
16:07 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host ganeti1026.eqiad.wmnet with OS buster [production]
16:07 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host ganeti1025.eqiad.wmnet with OS buster [production]
16:05 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host kubernetes1022.eqiad.wmnet with OS bullseye [production]
16:00 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host prometheus1006.eqiad.wmnet with OS bullseye [production]
15:59 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
15:58 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
15:56 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host prometheus1005.eqiad.wmnet with OS bullseye [production]
15:54 <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.38.0-wmf.13/extensions/Wikibase/client/data-bridge/: Backport: [[gerrit:747701|bridge: fix terms of service and copyright missing]] (duration: 01m 06s) [production]
15:42 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
15:42 <elukey> shutdown kafka-main2002 for BIOS+NIC firmware upgrades [production]
15:40 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
15:36 <ladsgroup@deploy1002> Synchronized wmf-config/CommonSettings.php: Config: [[gerrit:747857|Gradual roll out of $wgMaxExecutionTimeForExpensiveQueries (T297708)]] (duration: 01m 06s) [production]
15:35 <ladsgroup@deploy1002> Synchronized php-1.38.0-wmf.12/includes/: Backport: [[gerrit:747696|Allow setting max execution time to several special pages (T297708)], Part II (duration: 01m 11s) [production]
15:34 <ladsgroup@deploy1002> Synchronized php-1.38.0-wmf.12/includes/DefaultSettings.php: Backport: [[gerrit:747696|Allow setting max execution time to several special pages (T297708)], Part I (duration: 01m 05s) [production]
15:32 <ladsgroup@deploy1002> Synchronized php-1.38.0-wmf.13/includes/: Backport: [[gerrit:747695|Allow setting max execution time to several special pages (T297708)], Part II (duration: 01m 12s) [production]
15:30 <ladsgroup@deploy1002> Synchronized php-1.38.0-wmf.13/includes/DefaultSettings.php: Backport: [[gerrit:747695|Allow setting max execution time to several special pages (T297708)], Part I (duration: 01m 06s) [production]
15:29 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
15:28 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
15:26 <volans@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host elastic1084.mgmt.eqiad.wmnet with reboot policy FORCED [production]
15:25 <volans@cumin1001> START - Cookbook sre.hosts.provision for host elastic1084.mgmt.eqiad.wmnet with reboot policy FORCED [production]
15:24 <volans@cumin1001> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host elastic1084.mgmt.eqiad.wmnet with reboot policy FORCED [production]
15:24 <volans@cumin1001> START - Cookbook sre.hosts.provision for host elastic1084.mgmt.eqiad.wmnet with reboot policy FORCED [production]
15:22 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
15:21 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
15:19 <volans@cumin1001> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host elastic1084.mgmt.eqiad.wmnet with reboot policy FORCED [production]
15:19 <volans@cumin1001> START - Cookbook sre.hosts.provision for host elastic1084.mgmt.eqiad.wmnet with reboot policy FORCED [production]
15:15 <volans@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host elastic1084.mgmt.eqiad.wmnet with reboot policy FORCED [production]
15:07 <volans@cumin1001> START - Cookbook sre.hosts.provision for host elastic1084.mgmt.eqiad.wmnet with reboot policy FORCED [production]
15:05 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
15:04 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
15:03 <ladsgroup@deploy1002> Synchronized php-1.38.0-wmf.12/includes/libs/rdbms/database/: Backport: [[gerrit:747692|rdbms: add query timeout support to Database::select() (T129093 T195792)]] (duration: 01m 11s) [production]