|
2016-04-18
§
|
| 18:42 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Depool pc2005 (duration: 00m 26s) |
[production] |
| 18:37 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Repool pc2004 (duration: 00m 25s) |
[production] |
| 18:27 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool pc1004 (duration: 00m 36s) |
[production] |
| 17:31 |
<elukey> |
restarted kafkatee on oxygen after module upgrade (was stuck from this morning due to a kafka broker restart) |
[production] |
| 15:31 |
<andrewbogott> |
migration integration-slave-trusty-1018 to labvirt1009 |
[integration] |
| 14:51 |
<jzerebecki> |
reloading zuul for d9e80bd..113ea21 |
[releng] |
| 14:21 |
<ottomata> |
upgraded pykafka on hafnium and restarted statsv |
[production] |
| 14:04 |
<andrewbogott> |
puppet agent -tv on labvirt1002 which restarted nrpe and resolved icinga alerts |
[production] |
| 14:02 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool pc1004 (duration: 00m 29s) |
[production] |
| 14:01 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Depool pc2004 (duration: 00m 27s) |
[production] |
| 12:59 |
<jzerebecki> |
integration-slave-trusty-1014:~$ sudo -i rm -f /mnt/jenkins-workspace/workspace/mediawiki-extensions-hhvm/src/extensions/GlobalCssJs/.git/refs/heads/wmf/1.27.0-wmf.11.lock |
[releng] |
| 12:57 |
<jzerebecki> |
integration-slave-trusty-1014:~$ sudo -i 'rm -f /mnt/jenkins-workspace/workspace/mediawiki-extensions-hhvm/src/extensions/GlobalCssJs/.git/refs/heads/wmf/1.27.0-wmf.11.lock' |
[releng] |
| 11:30 |
<mobrovac> |
restbase restarting eqiad after https://gerrit.wikimedia.org/r/#/c/283958/ |
[production] |
| 10:11 |
<elukey> |
execute sudo eventloggingctl restart on eventlogging1001 |
[analytics] |
| 10:07 |
<_joe_> |
traffic from eqiad caches switched to codfw for restbase,citoid,cxserver |
[production] |
| 09:43 |
<moritzm> |
installing openssh updates on jessie systems |
[production] |
| 09:20 |
<_joe_> |
upgrading HHVM on terbium |
[production] |
| 09:19 |
<godog> |
depool restbase100[56].eqiad.wmnet, about to get decomissioned |
[production] |
| 09:17 |
<godog> |
repool restbase2006 after raid expansion |
[production] |
| 09:08 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Update master comments in preparation for dc failover (duration: 00m 26s) |
[production] |
| 09:07 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Update master comments in preparation for dc failover (duration: 01m 41s) |
[production] |
| 07:02 |
<jynus> |
changing binlog_format to MIXED on db2018 |
[production] |
|
2016-04-16
§
|
| 20:57 |
<bd808> |
Restarted to add bot to #wikimedia-codereview |
[tools.stashbot] |
| 14:25 |
<andrewbogott> |
rebooting mw1132 — OOM |
[production] |
| 14:24 |
<andrewbogott> |
rebooting mw1134 — OOM |
[production] |
| 14:20 |
<Krenair> |
restarted qa-morebots per request |
[releng] |
| 14:18 |
<Krenair> |
<jzerebecki> !log reloading zuul for 3f64dbd..c6411a1 |
[releng] |
| 13:58 |
<jzerebecki> |
reloading zuul for 3f64dbd..c6411a1 |
[releng] |
| 10:34 |
<jynus> |
recreating _counters table on s3 to solve replication issues |
[production] |
| 10:08 |
<jynus> |
recreating hitcounter table on s3 to solve replication issues |
[production] |
| 09:31 |
<volans> |
Skipped on dbstore1002 query from replica: 'DELETE FROM `bawiktionary`.`hitcounter`' |
[production] |
| 09:31 |
<jynus> |
importing sys schema to db1075 |
[production] |
| 06:44 |
<akosiaris> |
restart fermium to apply extra vcpus assignment |
[production] |
| 06:23 |
<volans> |
Stopped and restarted replica on dbstore2002 for s3 to "unstuck" the replica |
[production] |
| 02:31 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Sat Apr 16 02:31:33 UTC 2016 (duration 9m 11s) |
[production] |
| 02:22 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.21) (duration: 09m 03s) |
[production] |
| 02:16 |
<urandom> |
Bootstraping restbase1009-a.eqiad.wmnet : T95253 |
[production] |
|
2016-04-15
§
|
| 22:28 |
<mutante> |
rebooting bast4001 - debugging install issue |
[production] |
| 21:04 |
<volans@tin> |
Synchronized wmf-config/db-eqiad.php: Repool new db1075,1077,1078 after TLS upgrade on s3 - T111654 (duration: 00m 36s) |
[production] |
| 16:08 |
<demon@tin> |
Synchronized wmf-config/CommonSettings.php: more liberal initialisesettings invalidation (duration: 00m 42s) |
[production] |
| 15:34 |
<godog> |
remove cassandra metrics for restbase100[1234]* restbase100[789] restbase2004 - T132771 |
[production] |
| 14:40 |
<gehel> |
reindexing all wikis after the switch to codfw (T132762) |
[production] |
| 14:06 |
<urandom> |
start decommission of restbase1009-a.eqiad.wmnet : T95253 |
[production] |
| 12:15 |
<gehel> |
increase max request size for elasticsearch |
[production] |
| 12:14 |
<volans> |
Re-arrange s3 replica topology: making codfw replicate from db1075 (this time for real) - T111654 |
[production] |
| 12:08 |
<bblack> |
experimenting on carbon HTTP config (apt/mirrors/ubuntu.wm.o) - watch out for installer / package-update issues! |
[production] |