|
2018-12-12
§
|
| 11:03 |
<volans> |
restarting Icinga with debug log on icinga1001 |
[production] |
| 10:51 |
<mobrovac@deploy1001> |
Finished deploy [restbase/deploy@44e0955]: Bring restbase201[3-8] up to date, try #2b - T211416 (duration: 10m 11s) |
[production] |
| 10:41 |
<mobrovac@deploy1001> |
Started deploy [restbase/deploy@44e0955]: Bring restbase201[3-8] up to date, try #2b - T211416 |
[production] |
| 10:40 |
<mobrovac@deploy1001> |
Finished deploy [restbase/deploy@44e0955]: Bring restbase201[3-8] up to date, try #2 - T211416 (duration: 00m 15s) |
[production] |
| 10:40 |
<mobrovac@deploy1001> |
Started deploy [restbase/deploy@44e0955]: Bring restbase201[3-8] up to date, try #2 - T211416 |
[production] |
| 10:26 |
<volans> |
Icinga is having issue restarting properly, investigation ongoing |
[production] |
| 10:08 |
<banyek> |
executing schema change on db1070 (s5 master) - T85757 |
[production] |
| 09:40 |
<banyek> |
repooling labsdb1010 - T210693 |
[production] |
| 09:34 |
<banyek@deploy1001> |
Synchronized wmf-config/db-eqiad.php: T85757: repool db1082 (duration: 00m 52s) |
[production] |
| 09:29 |
<banyek> |
repooling db1082 - T85757 |
[production] |
| 09:25 |
<banyek> |
restarting replication on db1082 after schema change - T85757 |
[production] |
| 09:25 |
<banyek> |
fixing triggers on db1124:3315- T85757 |
[production] |
| 09:22 |
<banyek> |
executing schema change with replication on db1082 - T85757 |
[production] |
| 09:20 |
<banyek> |
stopping replication on db1082 for schema change - T85757 |
[production] |
| 09:15 |
<moritzm> |
installing pixman security updates on trusty (Debian already fixed) |
[production] |
| 08:48 |
<banyek@deploy1001> |
Synchronized wmf-config/db-eqiad.php: T85757: depool db1082 (duration: 00m 51s) |
[production] |
| 08:38 |
<banyek> |
depooling db1082 for schema change - T85757 |
[production] |
| 08:37 |
<marostegui> |
Remove old backup directory from db1116 - T206743 |
[production] |
| 08:18 |
<godog> |
decommissioning cassandra-b, restbase2006 -- T210843 |
[production] |
| 08:03 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Fully repool db1088 (duration: 00m 51s) |
[production] |
| 07:49 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Increase traffic for db1088 (duration: 00m 52s) |
[production] |
| 07:38 |
<marostegui> |
Deploy schema change on db2040 (s7 codfw master), this will generate lag on codfw T86338 T202167 |
[production] |
| 07:35 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Increase traffic for db1088 (duration: 00m 51s) |
[production] |
| 07:09 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Increase traffic for db1088 (duration: 00m 51s) |
[production] |
| 06:55 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Increase traffic for db1088 (duration: 00m 52s) |
[production] |
| 06:44 |
<marostegui> |
Stop MySQL on db1088 for mysql and kernel upgrade |
[production] |
| 06:44 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1088 for mysql upgrade (duration: 01m 07s) |
[production] |
| 06:37 |
<marostegui> |
Deploy schema change on s4 primary master (db1068) T86338 |
[production] |
| 06:00 |
<marostegui> |
Deploy schema change on s8 primary master (db1071) T86338 T202167 |
[production] |
| 00:47 |
<tgr@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:478467|Add some missing groups to the privileged list]] (duration: 00m 51s) |
[production] |
| 00:46 |
<tgr@deploy1001> |
Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:467110|Bring up password change logging to the same standards as login logging]] [[gerrit:478467|Add some missing groups to the privileged list]] (duration: 00m 53s) |
[production] |
|
2018-12-11
§
|
| 22:50 |
<chasemp> |
ssh to tar archive data from logstash1006 /mnt (external) to labstore1007 |
[production] |
| 22:06 |
<XioNoX> |
push loopback filter term return-tcp to all routers - T207962 |
[production] |
| 22:04 |
<urandom> |
decommissioning cassandra-a, restbase2006 -- T210843 |
[production] |
| 19:57 |
<XioNoX> |
apply BGP_IXP_RS_in and avoid HE to cr4-ulsfo - T211079 |
[production] |
| 19:28 |
<jforrester@deploy1001> |
Synchronized php-1.33.0-wmf.8/extensions/Flow/modules/engine/misc/flow-handlebars.js: T211707 Hot-deploy fix for StructuredDiscussions pages (duration: 00m 52s) |
[production] |
| 19:22 |
<jforrester@deploy1001> |
Synchronized php-1.33.0-wmf.6/extensions/VisualEditor/modules/ve-mw/init/ve.init.mw.Target.js: T209619: Hot-deploy Ibcf3c93e (duration: 00m 52s) |
[production] |
| 18:59 |
<mholloway-shell@deploy1001> |
Finished deploy [mobileapps/deploy@ecd5fb6]: fix site CSS URL (duration: 03m 58s) |
[production] |
| 18:55 |
<mholloway-shell@deploy1001> |
Started deploy [mobileapps/deploy@ecd5fb6]: fix site CSS URL |
[production] |
| 18:54 |
<XioNoX> |
push BGP_IXP_RS_in to all routers (but don't apply it to any peers, needs to be done manually) - T211079 |
[production] |
| 18:33 |
<cmjohnson1> |
swapping disk slot 0 db1063 |
[production] |
| 18:23 |
<XioNoX> |
push BGP_IXP_RS_in to cr2-eqord - T211079 |
[production] |
| 18:06 |
<anomie@deploy1001> |
Synchronized php-1.33.0-wmf.6/includes/user/User.php: Backport fix for T210621, for real this time (duration: 00m 53s) |
[production] |
| 18:00 |
<jforrester@deploy1001> |
Synchronized wmf-config/Wikibase.php: T211237 Hot-deploy disabling Wikibase federation, unused except in Beta Cluster (duration: 00m 52s) |
[production] |
| 17:33 |
<_joe_> |
restarting pybal on lvs2003 |
[production] |
| 17:23 |
<XioNoX> |
push changes tested on cr4-ulsfo to all routers - T211079 |
[production] |
| 17:16 |
<_joe_> |
restarting pybal on lvs2006 |
[production] |
| 17:15 |
<oblivian@puppetmaster1001> |
conftool action : set/weight=1; selector: service=search-omega-ssl |
[production] |
| 17:15 |
<oblivian@puppetmaster1001> |
conftool action : set/weight=1; selector: service=search-psi-ssl |
[production] |
| 17:15 |
<oblivian@puppetmaster1001> |
conftool action : set/pooled=yes; selector: service=search-psi-ssl |
[production] |