3501-3550 of 10000 results (41ms)
2021-08-16 ยง
18:17 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
18:15 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
17:52 <cmjohnson@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on mw1453.eqiad.wmnet with reason: REIMAGE [production]
17:49 <cmjohnson@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1453.eqiad.wmnet with reason: REIMAGE [production]
17:34 <cmjohnson1> installing new line card in slot1 cr2-eqiad T277339 [production]
17:33 <ladsgroup@deploy1002> Synchronized php-1.37.0-wmf.18/extensions/SpamBlacklist/includes/SpamBlacklistHooks.php: Backport: [[gerrit:712965|Try to use EditStash before re-rendering (T288639)]] (duration: 00m 59s) [production]
17:30 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
17:28 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
17:25 <XioNoX> cr1-eqiad> request chassis fpc offline slot 5 - T277339 [production]
17:17 <cmjohnson1> installing new line card in slot1 cr1-eqiad T277339 [production]
17:11 <ejegg> updated fundraising CiviCRM from f3895dc907 to 1e32084622 [production]
17:08 <XioNoX> asw2-a-eqiad> request virtual-chassis vc-port set pic-slot 1 member 8 port 1 - T288834 [production]
17:05 <XioNoX> asw2-a-eqiad> request virtual-chassis vc-port delete pic-slot 1 member 8 port 1 - T288834 [production]
16:40 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
16:37 <cwhite> restart logstash on logstash1008 [production]
16:36 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
16:01 <mutante> LDAP - added user tandic to nda group (T288527) [production]
15:37 <ryankemper> [WDQS] Re-pooled `codfw`: `ryankemper@puppetmaster1001:~$ sudo -i confctl --quiet --object-type discovery select 'dnsdisc=wdqs,name=codfw' set/pooled=true` [production]
14:42 <mutante> miscweb - deploying new microsite for Wikidata Query Builder subpage (T266703) [production]
14:41 <mutante> mw1455 - works fine after a reimage, unknown why it didnt last time, but ok :) [production]
14:11 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1455.eqiad.wmnet with reason: REIMAGE [production]
14:08 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1455.eqiad.wmnet with reason: REIMAGE [production]
13:53 <mutante> mw1455 - mysteriously showing a bunch of issues in icinga, broken packages, envoy, memcached etc, after recent fresh install, trying another reimage (T273915) [production]
13:50 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
13:48 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
13:42 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:711515|Remove $wmgWikibaseFineGrainedLuaTracking (T288612)]] (duration: 00m 58s) [production]
13:42 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
13:40 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5 days, 8:00:00 on planet1002.eqiad.wmnet with reason: known issue [production]
13:40 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 5 days, 8:00:00 on planet1002.eqiad.wmnet with reason: known issue [production]
13:40 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
13:40 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/Wikibase.php: Config: [[gerrit:711514|Stop setting $wgWBClientSettings['fineGrainedLuaTracking'] (T288612)]] (duration: 00m 58s) [production]
13:37 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/InitialiseSettings-labs.php: Config: [[gerrit:711513|Remove $wmgWikibaseClientUseTermsTableSearchFields (T288612)]] (beta, 2/2) (duration: 00m 59s) [production]
13:36 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:711513|Remove $wmgWikibaseClientUseTermsTableSearchFields (T288612)]] (prod, 1/2) (duration: 00m 59s) [production]
13:34 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
13:33 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/Wikibase.php: Config: [[gerrit:711512|Stop setting 'useTermsTableSearchFields' Wikibase option (T288612)]] (duration: 00m 59s) [production]
13:32 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
12:22 <Lucas_WMDE> EU backport+config window done (slightly belatedly) [production]
12:20 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
12:19 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
12:18 <tstarling@deploy1002> Synchronized php-1.37.0-wmf.18/extensions/SecurePoll/includes/Pages/VotePage.php: allow linking by title (duration: 00m 58s) [production]
12:17 <jynus@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2099.codfw.wmnet with reason: REIMAGE [production]
12:15 <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.37.0-wmf.18/extensions/Math/src/HookHandlers/ParserHooksHandler.php: Backport: [[gerrit:712962|Support null content in parser tag hook (T288846)]] (hopefully also fixes T288790) (duration: 00m 59s) [production]
12:15 <jynus@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on db2099.codfw.wmnet with reason: REIMAGE [production]
12:14 <kormat> clean up old /root/.my.cnf files T150446 [production]
11:54 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:50 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:49 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:712754|Add extendedconfirmed on zhwiki (T287322)]] + Config: [[gerrit:713255|Fix extendedconfirmed for bots on zhwiki (T287322)]] (duration: 01m 01s) [production]
11:44 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:42 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:26 <Lucas_WMDE> namespaceDupes.php for T287024 finished [production]