|
2017-11-24
§
|
| 12:50 |
<jynus> |
resetting replication on es1011 for consistency with other replica sets |
[production] |
| 12:40 |
<jynus> |
setting up s8 topology on eqiad |
[production] |
| 12:38 |
<jynus> |
disable puppet on db1071 and stop local s5 heartbeat there |
[production] |
| 12:32 |
<reedy@tin> |
Synchronized docroot/mediawiki/keys/: Fixup keys (duration: 00m 45s) |
[production] |
| 12:13 |
<marostegui> |
Enable GTID on es2018 - T181293 |
[production] |
| 11:57 |
<marostegui> |
Disable puppet on es2018 - T181293 |
[production] |
| 11:50 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: depool es2018 T181293 (duration: 00m 45s) |
[production] |
| 11:48 |
<marostegui> |
Reboot es2018 after full-upgrade - T181293 |
[production] |
| 11:25 |
<marostegui> |
Restart mysql on es2018 |
[production] |
| 11:25 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: db2085:3318, db2086:3318 (duration: 00m 43s) |
[production] |
| 11:24 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Pool db2038, db2085:3318, db2086:3318 (duration: 00m 45s) |
[production] |
| 10:55 |
<marostegui> |
Restart MySQL on db2086 to move s5 to s8 |
[production] |
| 10:37 |
<jynus> |
cancelling db2085 restart, only doing mysql:s5 |
[production] |
| 10:35 |
<jynus> |
restarting db2085 (including both s5 and s3 instances) |
[production] |
| 10:33 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool all future s8 slaves for a topology change - T177208 (duration: 00m 45s) |
[production] |
| 10:22 |
<moritzm> |
installing ca-cerfificates updates on trusty hosts |
[production] |
| 09:57 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase traffic for db1097:3315 and db1092 (duration: 00m 45s) |
[production] |
| 09:50 |
<jynus> |
restarting db2045 |
[production] |
| 09:32 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase traffic for db1101:3318 db1097:3315 abd db1092 (duration: 00m 45s) |
[production] |
| 08:49 |
<marostegui> |
Stop MySQL on db1092 |
[production] |
| 08:48 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase traffic for db1101:3318 in s5 to warm it up and depool db1092 - T178359 T177208 (duration: 00m 45s) |
[production] |
| 08:40 |
<moritzm> |
installing java security updates on notebook* hosts |
[production] |
| 08:38 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Slowly pool db1097:3315 - T178359 (duration: 00m 45s) |
[production] |
| 08:37 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Slowly pool db1097:3315 - T178359 (duration: 00m 45s) |
[production] |
| 08:20 |
<moritzm> |
installing java security updates on meitnerium |
[production] |
| 08:15 |
<moritzm> |
installing java security updates on stat1004 |
[production] |
| 08:14 |
<hashar> |
restarting jenkins on contint1001 for a java update |
[production] |
| 08:07 |
<elukey> |
re-enabling piwik on bohrium (only VM running on ganeti1006 atm) after mysql tables restore completed |
[production] |
| 06:47 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Slowly pool db1101:3318 in s5 to warm it up - T178359 (duration: 00m 45s) |
[production] |
| 06:46 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Slowly pool db1101:3318 in s5 to warm it up - T178359 (duration: 00m 49s) |
[production] |
| 00:12 |
<reedy@tin> |
Synchronized docroot/mediawiki/keys/: Add Brian Wolff's key (duration: 00m 45s) |
[production] |
|
2017-11-23
§
|
| 23:42 |
<reedy@tin> |
Synchronized wmf-config/: Simplify variable assignment (duration: 00m 47s) |
[production] |
| 23:25 |
<reedy@tin> |
Synchronized composer.lock: (no justification provided) (duration: 00m 44s) |
[production] |
| 23:24 |
<reedy@tin> |
Synchronized composer.json: (no justification provided) (duration: 00m 45s) |
[production] |
| 23:23 |
<reedy@tin> |
Synchronized phpcs.xml: (no justification provided) (duration: 00m 45s) |
[production] |
| 22:39 |
<demon@tin> |
Synchronized wmf-config/: style fixes, no-op (duration: 00m 47s) |
[production] |
| 22:19 |
<demon@tin> |
Synchronized wmf-config/CommonSettings.php: no-op, moving stuff around (duration: 00m 47s) |
[production] |
| 21:34 |
<ariel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=mw2251.codfw.wmnet |
[production] |
| 21:15 |
<apergos> |
rebooted mw2251 after unresponsive on mgmt console and no ping |
[production] |
| 20:49 |
<demon@tin> |
Synchronized static/images/project-logos/nowikimedia.png: logo update (duration: 02m 51s) |
[production] |
| 17:29 |
<akosiaris> |
set ganeti1006 as drained. ganeti1005 was already set. That will prevent scheduling VMs on those. T181121 |
[production] |
| 17:01 |
<akosiaris> |
force powercycle on ganeti1006 T181121 |
[production] |
| 16:30 |
<akosiaris> |
gnt-node failover ganeti1006 |
[production] |
| 15:50 |
<jynus> |
disable puppet on db2023, db2038 for deployment whle transfer is ongoing |
[production] |
| 15:23 |
<moritzm> |
rebooting mwdebug* servers for update to 4.9.51 |
[production] |
| 14:10 |
<marostegui> |
Stop Mysql on db1101.s5 to clone db1097 - T178359 |
[production] |
| 14:09 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Pool db1053 in s2 as vslow to replace db1021 - T134476 (duration: 00m 45s) |
[production] |
| 13:13 |
<jynus> |
sutting down db2023 and db2038 for cloning |
[production] |
| 12:47 |
<reedy@tin> |
Synchronized wmf-config/CommonSettings.php: GWToolset migratory config T87928 (duration: 00m 46s) |
[production] |
| 12:40 |
<moritzm> |
migrating instances off ganeti2001 / kernel reboot to 4.9.51 |
[production] |