501-550 of 10000 results (27ms)
2019-07-08 §
07:34 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
07:34 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
07:00 <elukey> add base::firewall to stat1004 - T170826 [production]
06:43 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Fully repool db1109 after changing its binlog format (duration: 00m 49s) [production]
06:36 <marostegui> Run compare for s5 main tables on db2038 vs db2059 - T221533 [production]
06:09 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More traffic to db1109 after changing its binlog format (duration: 00m 49s) [production]
05:50 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Fully repool db1094 after upgrade, slowly repool db1109 after changing its binlog format (duration: 00m 49s) [production]
05:45 <marostegui> Restart MySQL on db1109 to pick up STATEMENT as binlog format - T227062 [production]
05:44 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1109 for binlog format change (duration: 00m 49s) [production]
05:39 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More weight to db1094 after upgrade (duration: 00m 51s) [production]
05:31 <marostegui> Compress medium wikis on labsdb1009 - T222978 [production]
05:24 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1094 after upgrade (duration: 00m 49s) [production]
05:22 <marostegui> Drop empty table edit_page_tracking from some s3 wikis - T57385 [production]
05:11 <marostegui> Drop empty table edit_page_tracking from s7 - T57385 [production]
05:08 <marostegui> Stop MySQL on db1094 for upgrade [production]
05:08 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1094 for upgrade (duration: 00m 50s) [production]
03:19 <sbassett@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Temporary make account creation limits more restrictive (duration: 00m 53s) [production]
01:16 <sbassett@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Temporary make account creation limits more restrictive (duration: 00m 50s) [production]
2019-07-07 §
20:13 <urandom> decommissioning restbase1017-b -- T222960 [production]
17:25 <urandom> decommissioning restbase1017-a -- T222960 [production]
15:14 <godog> power reset restbase2009 [production]
2019-07-06 §
07:56 <thcipriani> restarting gerrit out of heap space [production]
2019-07-05 §
17:18 <smalyshev@deploy1001> Finished deploy [wdqs/wdqs@4b7cdf5]: Deploy new versions preparing for reload (duration: 00m 39s) [production]
17:17 <smalyshev@deploy1001> Started deploy [wdqs/wdqs@4b7cdf5]: Deploy new versions preparing for reload [production]
17:17 <smalyshev@deploy1001> Finished deploy [wdqs/wdqs@4b7cdf5]: Deploy new versions preparing for reload (duration: 00m 01s) [production]
17:17 <smalyshev@deploy1001> Started deploy [wdqs/wdqs@4b7cdf5]: Deploy new versions preparing for reload [production]
15:32 <fsero> uploaded debian buster base docker image [production]
15:30 <jmm@cumin2001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
15:23 <fsero> restarting swift-container-sync on swift backends [production]
15:20 <jmm@cumin2001> START - Cookbook sre.ganeti.makevm [production]
15:15 <jmm@cumin2001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
15:05 <jmm@cumin2001> START - Cookbook sre.ganeti.makevm [production]
15:01 <jmm@cumin2001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
14:51 <jmm@cumin2001> START - Cookbook sre.ganeti.makevm [production]
14:24 <jmm@cumin2001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
14:15 <jmm@cumin2001> START - Cookbook sre.ganeti.makevm [production]
13:44 <elukey> roll restart of aqs on aqs100* to pick up new druid settings [production]
13:33 <fsero> disabling puppet on swift backends [production]
13:26 <fsero> restarting swift-container-sync on swift backends [production]
13:05 <ema> pool cp1090 w/ ATS backend T226638 [production]
12:12 <ema> depool cp1090 and reimage as upload_ats T226638 [production]
11:46 <ema> pool cp1088 w/ ATS backend T226638 [production]
11:39 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:39 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:38 <jijiki> Reboot ms-be1021 - T141756 - T227076 [production]
11:32 <jijiki> Upgrading smartarray firmware on ms-be1021 - T141756 - T227076 [production]
11:31 <moritzm> installing postgresql-9.4 updates on jessie [production]
11:10 <jmm@cumin1001> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) [production]
11:09 <jmm@cumin1001> START - Cookbook sre.ganeti.makevm [production]
11:05 <jmm@cumin2001> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) [production]