|
2018-09-26
ยง
|
| 13:34 |
<marostegui> |
Stop replication on s4 eqiad master for maintenance, lag will be generated |
[production] |
| 13:32 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2071 (duration: 00m 55s) |
[production] |
| 13:24 |
<zhuyifei1999_> |
restarted mariadb on -db-01 after max_allowed_packet fix |
[quarry] |
| 13:12 |
<marostegui> |
Deploy schema change on db2071 - T203709 |
[production] |
| 13:12 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Depool db2071 (duration: 00m 56s) |
[production] |
| 13:03 |
<moritzm> |
installing tomcat8 security updates |
[production] |
| 12:33 |
<marostegui> |
Deploy schema change on s4 eqiad, will generate lag - T203709 |
[production] |
| 11:57 |
<Amir1> |
[[gerrit:462927]] (ores) is going to beta |
[releng] |
| 11:43 |
<zeljkof> |
EU SWAT finished |
[production] |
| 11:42 |
<zfilipin@deploy1001> |
Synchronized wmf-config/throttle.php: SWAT: [[gerrit:462928|Lift account creation IP cap for 2018-09-26 (T205529)]] (duration: 00m 56s) |
[production] |
| 11:30 |
<zfilipin@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:462191|Enable VisualEditor in Project namespace on srwiki (T205206)]] (duration: 00m 57s) |
[production] |
| 11:11 |
<volans> |
disabled puppet on einsteinium to test ircecho failure, I'll re-enable it in max ~30m |
[production] |
| 10:47 |
<Amir1> |
1b5efba is being deployed |
[ores] |
| 10:40 |
<arturo> |
T205524 all sorts of restarts in all neutron daemons |
[admin] |
| 10:37 |
<Amir1> |
1b5efba is going to staging |
[ores] |
| 10:30 |
<volans> |
started ircecho on einsteinium |
[production] |
| 10:27 |
<volans> |
stop ircecho on einsteinium to register the nickname |
[production] |
| 10:26 |
<moritzm> |
rebooting mw1270-mw1290 for kernel security updates |
[production] |
| 10:20 |
<arturo> |
T205524 stop/start all neutron agents in cloudnet1003.eqiad.wmnet |
[admin] |
| 10:13 |
<arturo> |
T205524 restart all agents in cloudnet1004.eqiad.wmnet |
[admin] |
| 10:13 |
<_joe_> |
reenabling puppet on the MediaWiki hosts |
[production] |
| 10:10 |
<arturo> |
restart neutron-server in cloudcontrol1003, investigating T205524 |
[admin] |
| 10:08 |
<moritzm> |
rebooting conf1006 for kernel security update |
[production] |
| 09:51 |
<moritzm> |
rebooting conf1005 for kernel security update |
[production] |
| 09:46 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2092 (duration: 00m 56s) |
[production] |
| 09:45 |
<moritzm> |
rebooting conf1004 for kernel security update |
[production] |
| 09:29 |
<jynus> |
restarting icinga at einsteinium, unresponsive to commands |
[production] |
| 09:18 |
<arturo> |
T204745 migrating project to eqiad1 |
[wikimania-support] |
| 09:06 |
<marostegui> |
Deploy schema change on db2092 - T203709 |
[production] |
| 09:06 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Depool db2092 (duration: 00m 56s) |
[production] |
| 08:58 |
<_joe_> |
disabling puppet on all hosts with a MediaWiki setup before merging https://gerrit.wikimedia.org/r/c/operations/puppet/+/453093 |
[production] |
| 08:35 |
<onimisionipe> |
restarting blazegraph and updater on wdqs2003 |
[production] |
| 08:24 |
<hashar> |
Restarting CI Jenkins on contint1001 [#2] |
[production] |
| 08:24 |
<hashar> |
Restarting CI Jenkins on contint1001 [#2] |
[releng] |
| 08:22 |
<elukey> |
start mysql consumers on eventlog1002 after maintenance |
[analytics] |
| 08:22 |
<elukey> |
start eventlogging_sync on db1108 and the mysql kafka consumers on eventlog1002 after db1107 maintenance |
[production] |
| 08:21 |
<moritzm> |
rebooting mw1250-mw1269 for kernel security updates |
[production] |
| 08:17 |
<banyek> |
db1107 upgrade finished (T205288) |
[production] |
| 08:15 |
<banyek> |
db1107 upgrade finished |
[production] |
| 08:14 |
<hashar> |
Restarting CI Jenkins on contint1001 |
[releng] |
| 08:14 |
<hashar> |
Restarting CI Jenkins on contint1001 |
[production] |
| 08:04 |
<elukey> |
stop eventlogging_sync on db1108 and the mysql kafka consumers on eventlog1002 as prep step for db1107 maintenance |
[production] |
| 08:04 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1092, server crashed - T205514 (duration: 00m 56s) |
[production] |
| 08:00 |
<banyek> |
donwtiming db1107 as upgrade (kernel & mariadb) |
[production] |
| 07:51 |
<elukey> |
stop mysql consumers on eventlog1002 as prep step for db maintenance |
[analytics] |
| 07:50 |
<marostegui> |
Hard reset db1092, server crashed - T205514 |
[production] |
| 07:48 |
<addshore@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: [[gerrit:462864|Set testwikidatawiki CacheEpoch to 2018-09-19]] T205330 (duration: 00m 56s) |
[production] |
| 07:47 |
<elukey> |
reboot an-master1002 as attempt to clear out some systemd@fsck alarms |
[production] |
| 07:46 |
<godog> |
repair /dev/sdd1 on ms-be1043 - T199198 |
[production] |
| 07:41 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2085:3311 (duration: 00m 56s) |
[production] |