|
2016-06-20
§
|
| 12:56 |
<gehel> |
installing maps1001.eqiad.wmnet (secondary cluster, no traffic there yet) - T138092 |
[production] |
| 12:56 |
<paravoid> |
deactivating peerings with Telia Carrier/AS1299 (eqiad/codfw/ulsfo) |
[production] |
| 12:41 |
<moritzm> |
rebooting ms1001 for update to Linux 4.4 |
[production] |
| 12:13 |
<Amir1> |
started deploying ores in scb2001 bdc1e2bd |
[production] |
| 11:36 |
<godog> |
roll-restart swift on ms-be1* to apply https://gerrit.wikimedia.org/r/294691 |
[production] |
| 11:27 |
<Amir1> |
for ores in scb nodes |
[production] |
| 11:27 |
<Amir1> |
rollbacking ae71d842dfc0958e06922062dd09d49243332a6a |
[production] |
| 11:13 |
<_joe_> |
restarting uwsgi orse service |
[production] |
| 10:58 |
<Amir1> |
deploying bdc1e2b in ores nodes |
[production] |
| 10:53 |
<godog> |
roll-restart swift on ms-be2* to apply https://gerrit.wikimedia.org/r/294691 |
[production] |
| 10:44 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1071 completelly (duration: 00m 25s) |
[production] |
| 10:35 |
<jynus> |
db1071 stop, backup and reimage |
[production] |
| 10:31 |
<mobrovac> |
restbase started mobile-sections dump for eswiki on restbase1009 for T136964 |
[production] |
| 10:05 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1073 at 100% weight; depool db1071 for reimaging (duration: 00m 27s) |
[production] |
| 09:50 |
<moritzm> |
rolling reboot of restbase2001/restbase2002 for upgrade to Linux 4.4 |
[production] |
| 08:57 |
<Amir1> |
deploying 5dfe738 in ores nodes |
[production] |
| 08:15 |
<moritzm> |
installing libxlst security updates |
[production] |
| 07:43 |
<gehel> |
rebalancing shards on elasticsearch eqiad cluster |
[production] |
| 06:47 |
<_joe_> |
activating the jessie jobrunner, mw1299 |
[production] |
| 05:57 |
<ori@tin> |
Synchronized wmf-config/CommonSettings.php: Id5804a80: Better cache headers for 'Powered by MediaWiki' badge (2/2) (duration: 00m 35s) |
[production] |
| 05:56 |
<ori@tin> |
Synchronized static/images: Id5804a80: Better cache headers for 'Powered by MediaWiki' badge (1/2) (duration: 00m 33s) |
[production] |
| 02:29 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Mon Jun 20 02:29:01 UTC 2016 (duration 5m 44s) |
[production] |
| 02:23 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.6) (duration: 09m 54s) |
[production] |
|
2016-06-17
§
|
| 21:21 |
<urandom> |
Reenabling puppet and resetting configuration on xenon.eqiad.wmnet : T137419 |
[production] |
| 20:39 |
<urandom> |
Restarting Cassandra on xenon.eqiad.wmnet to apply -XX:+PreserveFramePointer : T137419 |
[production] |
| 20:35 |
<urandom> |
Disabling puppet on xenon.eqiad.wmnet : T137419 |
[production] |
| 20:23 |
<maxsem@tin> |
Synchronized php-1.28.0-wmf.6/extensions/WikimediaEvents/: https://gerrit.wikimedia.org/r/#/c/294958/ (duration: 00m 33s) |
[production] |
| 18:56 |
<urandom> |
Restarting Cassandra on xenon.eqiad.wmnet with -XX:+PreserveFramePointer : T137419 |
[production] |
| 18:32 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1073 with low weight after reimage (duration: 00m 35s) |
[production] |
| 16:29 |
<moritzm> |
installing squid security updates on carbon |
[production] |
| 15:59 |
<urandom> |
Starting html dumps from xenon.eqiad.wmnet and cerium.eqiad.wmnet : T137419 |
[production] |
| 15:54 |
<urandom> |
Restarting Cassandra on xenon.eqiad.wmnet to enable large pages : T137419 |
[production] |
| 14:55 |
<mobrovac> |
scb disabling puppet for stopping change-prop to clear transclusion queues |
[production] |
| 14:16 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Increase db1072 weight after repooling (duration: 00m 36s) |
[production] |
| 12:57 |
<jynus> |
stopping, backuping and reimaging db1073 |
[production] |
| 12:49 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1072 with low weight, depool db1073 (duration: 00m 27s) |
[production] |
| 12:49 |
<moritzm> |
rolling reboot of mw1157-mw1160 into new kernels |
[production] |
| 12:27 |
<moritzm> |
restarted hhvm on mw1133 and mw1135 |
[production] |
| 11:14 |
<moritzm> |
stopping puppet on hosts using service::node (restbase, sca, scb, aqs) for step-by-step rollout of two puppet patches for firejail/service::node |
[production] |
| 09:31 |
<_joe_> |
powercycling mw1140, OOMd |
[production] |
| 09:30 |
<moritzm> |
rolling reboot of mw1153,mw1155,mw1156 into new kernels |
[production] |
| 08:29 |
<hashar> |
Restarting Jenkins on gallium. Web interface at least is deadlocked somehow |
[production] |
| 07:23 |
<jynus> |
backuping and reimaging db1072 |
[production] |
| 07:18 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1072 for maintenance (duration: 00m 31s) |
[production] |