4301-4350 of 10000 results (77ms)
2019-05-28 §
16:57 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:49 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:49 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:41 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:41 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:35 <hoo> Ran scap pull on mw1240 (curl -H 'Host: www.wikidata.org' … mw1240.eqiad.wmnet/wiki/Special:SetEntitySchemaLabelDescriptionAliases/E10/en returned 404) [production]
16:27 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:27 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:20 <Lucas_WMDE> lucaswerkmeister-wmde@mw1271:~$ scap pull [production]
16:17 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:17 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
16:15 <moritzm> rearmed keyholder on deploy2001 following reboot [production]
16:09 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:09 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
16:09 <jmm@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
16:09 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
16:04 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:04 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:56 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:56 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:54 <papaul> shutting down db2091 for firmware upgrade [production]
15:53 <godog> put back wrongly-replaced sdf on ms-be2043 - T222654 [production]
15:42 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:42 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:42 <Lucas_WMDE> Extension:EntitySchema deployment finished successfully [production]
15:38 <Lucas_WMDE> lucaswerkmeister-wmde@mwmaint1002:~$ mwscript extensions/EntitySchema/maintenance/createPreexistingSchemas.php --wiki=wikidatawiki [production]
15:38 <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:512909|Enable extension EntitySchema in production]] (duration: 00m 56s) [production]
15:35 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:35 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:34 <lucaswerkmeister-wmde@deploy1001> Synchronized php-1.34.0-wmf.6/extensions/EntitySchema/: [[gerrit:512911|Steal maintenance script user]] (duration: 00m 58s) [production]
15:26 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:26 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:17 <Lucas_WMDE> lucaswerkmeister-wmde@mwmaint1002:~$ mwscript extensions/EntitySchema/maintenance/createPreexistingSchemas.php --wiki=testwikidatawiki [production]
15:17 <lucaswerkmeister-wmde@deploy1001> Synchronized php-1.34.0-wmf.7/extensions/EntitySchema/: [[gerrit:512912|Steal maintenance script user]] – forgot `git submodule update` before previous sync (duration: 00m 57s) [production]
15:15 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:15 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:11 <lucaswerkmeister-wmde@deploy1001> Synchronized php-1.34.0-wmf.7/extensions/EntitySchema/: [[gerrit:512912|Steal maintenance script user]] (duration: 00m 59s) [production]
15:08 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:07 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:01 <gehel@cumin2001> END (ERROR) - Cookbook sre.elasticsearch.rolling-reboot (exit_code=97) [production]
14:57 <jbond42> reboot ms-be2016 [production]
14:57 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:57 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:36 <gehel@cumin2001> START - Cookbook sre.elasticsearch.rolling-reboot [production]
14:30 <zfilipin@deploy1001> rebuilt and synchronized wikiversions files: group0 to 1.34.0-wmf.7 [production]
14:10 <herron> beginning rolling reboots of codfw kafka-main cluster for security updates [production]
14:10 <zfilipin@deploy1001> Finished scap: testwiki to php-1.34.0-wmf.7 and rebuild l10n cache (duration: 34m 22s) [production]
14:04 <gehel@cumin2001> END (ERROR) - Cookbook sre.elasticsearch.rolling-reboot (exit_code=97) [production]
13:50 <_joe_> hhvm restarted on mwdebug1001 [production]
13:48 <_joe_> stopping hhvm on mwdebug1001 for testing [production]