|
2018-10-11
ยง
|
| 22:56 |
<dzahn@neodymium> |
conftool action : set/pooled=inactive; selector: name=mwmaint1001.eqiad.wmnet |
[production] |
| 22:53 |
<mutante> |
netbox - correction, mwmaint1001 to status "Staged", following new lifecycle docs T192457 |
[production] |
| 22:50 |
<mutante> |
netbox - renamed mwmaint1001 to mw1279, changed status to inventory, renamed in DNS - T192457 |
[production] |
| 22:45 |
<krinkle@deploy1001> |
Synchronized php-1.32.0-wmf.24/includes/Revision/RenderedRevision.php: I553dba13486 (duration: 00m 51s) |
[production] |
| 22:30 |
<mutante> |
mwmaint1001 - shutting down after final backup of /home, renaming back to mw1297 in DNS and DHCP, and reinstalling (T192457) |
[production] |
| 21:53 |
<mutante> |
mwmaint1001 - schduled downtime, is being renamed back to mw1297 and reinstalled |
[production] |
| 21:47 |
<mutante> |
mwmaint2001 - rsyncing home dirs from mwmaint1002 to /root/home-mwmaint1002 (which includes home-terbium even!) in case anyone is missing anything from one of mwaint* |
[production] |
| 21:41 |
<mutante> |
mwmaint2001 - deleting 60G of unneeded files from home |
[production] |
| 20:37 |
<XioNoX> |
add IPv6 to mr1-ulsfo OOB - T206778 |
[production] |
| 18:46 |
<sbisson@deploy1001> |
Synchronized php-1.32.0-wmf.24/extensions/PageTriage/: SWAT: [[gerrit:465676|Handle page that are unnominated for deletion]] (duration: 00m 50s) |
[production] |
| 18:34 |
<sbisson@deploy1001> |
Synchronized php-1.32.0-wmf.24/extensions/PageTriage/modules/ext.pageTriage.views.list/ext.pageTriage.listControlNav.js: SWAT: [[gerrit:465677|Default to deleted and others when no type is selected on mode switch]] (duration: 00m 50s) |
[production] |
| 18:22 |
<sbisson@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:463319|Remove config for RCFilters variables being removed from Core]] (duration: 00m 49s) |
[production] |
| 18:14 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2083 and db2085:3318 (duration: 00m 48s) |
[production] |
| 18:13 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1101:3318 (duration: 00m 49s) |
[production] |
| 18:09 |
<sbisson@deploy1001> |
Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:466694|Add copyviobot group management to relevant wikis]] (duration: 00m 49s) |
[production] |
| 17:36 |
<gehel> |
repooling wdqs1003, catched up on lag |
[production] |
| 17:08 |
<banyek|away> |
automated binlog purging started on pc2004, pc2005, pc2006 |
[production] |
| 16:54 |
<gehel> |
depooling wdqs1003 to let it catch up on lag |
[production] |
| 15:38 |
<jynus@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1087 (duration: 00m 50s) |
[production] |
| 15:12 |
<marostegui> |
Stop MySQL on db2085:3318 to reclone db1101:3318 - T206743 |
[production] |
| 15:11 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1101:3318 (duration: 00m 49s) |
[production] |
| 15:04 |
<akosiaris> |
Media storage/Swift Swift set to active/passive |
[production] |
| 15:01 |
<akosiaris> |
Media storage/Swift Swift set to active/active |
[production] |
| 14:56 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1099:3318 (duration: 00m 48s) |
[production] |
| 14:52 |
<jynus> |
deploying wikidata row fix to db1087 with replication enabled |
[production] |
| 14:47 |
<END> |
(PASS) - Cookbook sre.switchdc.services.02-restore-ttl (exit_code=0) (volans@neodymium) |
[production] |
| 14:47 |
<START> |
- Cookbook sre.switchdc.services.02-restore-ttl (volans@neodymium) |
[production] |
| 14:36 |
<END> |
(PASS) - Cookbook sre.switchdc.services.01-switch-dc (exit_code=0) (volans@neodymium) |
[production] |
| 14:36 |
<Switching> |
services parsoid, restbase, restbase-async, mobileapps, apertium, citoid, cxserver, eventstreams, graphoid, mathoid, proton, pdfrender, recommendation-api, zotero, eventbus, ores, wdqs, wdqs-internal: codfw => eqiad (volans@neodymium) |
[production] |
| 14:36 |
<START> |
- Cookbook sre.switchdc.services.01-switch-dc (volans@neodymium) |
[production] |
| 14:35 |
<END> |
(PASS) - Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (exit_code=0) (volans@neodymium) |
[production] |
| 14:30 |
<banyek@deploy1001> |
Synchronized wmf-config/db-eqiad.php: T206743: mariadb: Depool db1087 (duration: 00m 49s) |
[production] |
| 14:30 |
<START> |
- Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (volans@neodymium) |
[production] |
| 14:28 |
<banyek> |
depooling db1087 (T206743) |
[production] |
| 14:28 |
<banyek> |
depooling db1087 |
[production] |
| 14:15 |
<elukey> |
reboot eventlog1002 for kernel upgrades |
[production] |
| 14:15 |
<jynus> |
applying row filling to (most) eqiad s8 dbs, including the mater |
[production] |
| 14:13 |
<moritzm> |
install libxml2 security updates on jessie servers |
[production] |
| 13:55 |
<jynus> |
recovering rows to db1092 |
[production] |
| 13:26 |
<jynus> |
filling in missing rows on dbstore1002 |
[production] |
| 13:23 |
<marostegui> |
Stop MySQL on db2083 to reclone db1116:3318 - T206743 |
[production] |
| 13:21 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Depool db2083 (duration: 00m 49s) |
[production] |
| 13:20 |
<marostegui> |
Stop MySQL on db1116:3318 to reclone it from db2083 - T206743 |
[production] |
| 12:43 |
<elukey> |
upgrade prometheus-memcached-exporter on mc1* |
[production] |
| 12:38 |
<elukey> |
upgrade prometheus-memcached-exporter on mc2* |
[production] |
| 12:15 |
<elukey> |
upgrade prometheus-memcached-exporter on mc2035 |
[production] |
| 12:14 |
<elukey> |
upload prometheus-memcached-exporter_0.4.1+git20181010.2fa99eb-1 to (jessie|stretch)-wikimedia |
[production] |
| 12:12 |
<Amir1> |
EU SWAT is done |
[production] |
| 11:30 |
<ladsgroup@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:466271|Set some small wikis to read new for change tag backend (T194164)]] (duration: 00m 50s) |
[production] |
| 11:10 |
<marostegui> |
Stop MYSQL on db2085:3318 and db1099:3318 T206743 |
[production] |