|
2018-02-13
ยง
|
| 17:12 |
<cmjohnson1> |
stat1001 going down to for rack relocation |
[production] |
| 17:04 |
<akosiaris@puppetmaster1001> |
conftool action : set/pooled=yes; selector: all (tags: ['dc=codfw', 'cluster=ores', 'service=ores']) |
[production] |
| 17:03 |
<akosiaris@puppetmaster1001> |
conftool action : set/pooled=yes; selector: all (tags: ['dc=eqiad', 'cluster=ores', 'service=ores']) |
[production] |
| 16:36 |
<demon@tin> |
Synchronized scap/plugins/clean.py: no-op, consistency (duration: 00m 55s) |
[production] |
| 16:23 |
<anomie@tin> |
Synchronized wmf-config/InitialiseSettings.php: Setting wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on group 0 (duration: 00m 56s) |
[production] |
| 16:17 |
<cmjohnson1> |
replacing disk poolcounte1002 |
[production] |
| 15:35 |
<marostegui> |
Deploy schema change on s5 codfw master (db2052), this will generate lag on codfw - T187089 T185128 T153182 |
[production] |
| 15:30 |
<bblack> |
deploying changes to URL-encoding normalization on caches - https://gerrit.wikimedia.org/r/407488 |
[production] |
| 15:20 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2066 (duration: 00m 55s) |
[production] |
| 15:01 |
<zeljkof> |
EU SWAT finished |
[production] |
| 14:59 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:406801|Update logo for urwikibooks, add hd logo (T185977)]] (duration: 00m 55s) |
[production] |
| 14:58 |
<zfilipin@tin> |
Synchronized static/images/project-logos/: SWAT: [[gerrit:406801|Update logo for urwikibooks, add hd logo (T185977)]] (duration: 00m 54s) |
[production] |
| 14:37 |
<zfilipin@tin> |
Synchronized static/images/project-logos/: SWAT: [[gerrit:409625|Change logos for sdwiki (T185865)]] (duration: 00m 55s) |
[production] |
| 14:25 |
<zfilipin@tin> |
Synchronized php-1.31.0-wmf.20/extensions/ContentTranslation/extension.json: SWAT: [[gerrit:410105|Add ext.cx.widgets.overlay dependency to template editor (T187119)]] (duration: 00m 55s) |
[production] |
| 14:22 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:408032|Add sitename for sdwiki (T184521)]] (duration: 00m 57s) |
[production] |
| 13:51 |
<marostegui> |
Reboot db2066 to pick up new kernel |
[production] |
| 13:50 |
<marostegui> |
Deploy schema change on dbstore2001 - T187089 T185128 T153182 |
[production] |
| 12:51 |
<mlitn@tin> |
Synchronized wmf-config/CommonSettings.php: Enable STL uploads on Commons (duration: 00m 56s) |
[production] |
| 12:20 |
<mlitn@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable STL uploads on Commons (duration: 00m 55s) |
[production] |
| 12:19 |
<mlitn@tin> |
Synchronized wmf-config/CommonSettings.php: Enable STL uploads on Commons (duration: 00m 55s) |
[production] |
| 12:07 |
<mlitn@tin> |
Synchronized php-1.31.0-wmf.20/extensions/3D/modules/ext.3d.js: Fix 3D badge (duration: 00m 56s) |
[production] |
| 11:57 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Depool db2066 (duration: 00m 55s) |
[production] |
| 11:56 |
<marostegui> |
Deploy schema change on db2066 - T187089 T185128 T153182 |
[production] |
| 11:50 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Rpool db2038 and db2059 (duration: 00m 55s) |
[production] |
| 11:47 |
<jynus> |
reenabling puppet on all eqiad databases |
[production] |
| 11:41 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1099 (duration: 00m 56s) |
[production] |
| 11:37 |
<marostegui> |
Stop MySQL on db2059 and db2038 for kernel upgrade |
[production] |
| 11:29 |
<ema> |
lvs1003: restart pybal to reconnect to etcd |
[production] |
| 11:27 |
<ema> |
lvs1006/1010: restart pybal to reconnect to etcd |
[production] |
| 11:26 |
<ema> |
lvs4005: restart pybal to reconnect to etcd |
[production] |
| 11:23 |
<ema> |
esams primary LVSs: restart pybal to reconnect to etcd |
[production] |
| 11:21 |
<ema> |
esams secondary LVSs: restart pybal to properly reconnect to etcd |
[production] |
| 11:14 |
<ema> |
repool cp3007 |
[production] |
| 11:13 |
<ema> |
depool cp3007 to test pybal's behavior on lvs3002 |
[production] |
| 10:51 |
<filippo@tin> |
Synchronized wmf-config/ProductionServices.php: depool poolcounter1002 for disk replacement (duration: 00m 56s) |
[production] |
| 10:28 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1099 (duration: 00m 54s) |
[production] |
| 10:08 |
<godog> |
roll-restart ms-fe in codfw/eqiad after applying https://gerrit.wikimedia.org/r/c/409942/ |
[production] |
| 10:03 |
<ema> |
restart pybal on lvs2003 |
[production] |
| 09:58 |
<ema> |
restart pybal on lvs2006 |
[production] |
| 09:52 |
<filippo@neodymium> |
conftool action : set/pooled=no; selector: name=ms-fe2005.codfw.wmnet |
[production] |
| 09:49 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2075, depool db2038 and db2059 (duration: 00m 55s) |
[production] |
| 09:32 |
<marostegui> |
Stop mysql on db2075 for mysql and kernel upgrade |
[production] |
| 09:30 |
<marostegui> |
Stop replication in sync on db1089 and dbstore1002 - T162807 |
[production] |
| 09:29 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1065 - T162807 (duration: 00m 54s) |
[production] |
| 09:22 |
<elukey> |
powercycle analytics1062 - not reachable via ssh, frozen via serial console |
[production] |
| 09:22 |
<jynus> |
disabling puppet on all eqiad databases |
[production] |
| 09:20 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1065 - T162807 (duration: 00m 55s) |
[production] |
| 09:20 |
<marostegui> |
Stop replication in sync on db1089 and db1065 - T162807 |
[production] |
| 09:12 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2084:3315, depool db2075 (duration: 00m 55s) |
[production] |
| 09:01 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1099:3311 - T162807 (duration: 00m 55s) |
[production] |