|
2015-09-14
§
|
| 18:05 |
<urandom> |
rebuilding restbase-test2001.codfw (nodetool rebuild -- eqiad) |
[production] |
| 16:12 |
<catrope@tin> |
Synchronized php-1.26wmf22/extensions/Echo/: For real this time (duration: 00m 11s) |
[production] |
| 16:06 |
<ottomata> |
stopping hdfs journalnode on analytics1011 to copy journal edits to new journalnodes on analytics1035 and analytics1052 |
[production] |
| 15:46 |
<godog> |
switch to openjdk-8 and bounce cassandra on restbase-test200* |
[production] |
| 15:39 |
<bblack> |
reinstalling lvs4003, lvs4004 (jessie upgrade: T96375) (typo earlier) |
[production] |
| 15:39 |
<bblack> |
reinstalling lvs4003, lvs4003 (jessie upgrade: T96375) |
[production] |
| 15:34 |
<catrope@tin> |
Synchronized php-1.26wmf22/extensions/Echo/: SWAT (duration: 00m 13s) |
[production] |
| 15:05 |
<krenair@tin> |
Synchronized .gitignore: https://gerrit.wikimedia.org/r/#/c/237529/ (duration: 00m 13s) |
[production] |
| 15:05 |
<krenair@tin> |
Synchronized docroot/noc: https://gerrit.wikimedia.org/r/#/c/237529/ (duration: 00m 12s) |
[production] |
| 15:04 |
<krenair@tin> |
Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/237529/ (duration: 00m 11s) |
[production] |
| 15:01 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/234980/ (duration: 00m 12s) |
[production] |
| 13:38 |
<godog> |
stop puppet on restbase-test2001 and turn up cassandra |
[production] |
| 12:56 |
<bblack> |
rebooting lvs2006 to test eth hw params stuff... |
[production] |
| 12:55 |
<krenair@tin> |
Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/238125/ (duration: 00m 13s) |
[production] |
| 12:50 |
<urandom> |
starting Cassandra repair on restbase1003 (nodetool repair -pr) |
[production] |
| 12:32 |
<godog> |
enable dc encryption on cassandra test cluster and rolling restart |
[production] |
| 11:33 |
<mobrovac> |
citoid deploying d569951 |
[production] |
| 10:35 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool es1002, es1005, es1008 (duration: 00m 12s) |
[production] |
| 10:04 |
<jynus> |
db1029 (x1-master) temporarily saturated by connections- flow was unresponsive for 10 minutes; migration partially aborted |
[production] |
| 09:08 |
<jynus> |
applying schema change to flowdb |
[production] |
| 08:52 |
<godog> |
rename cassandra test cluster and restart |
[production] |
| 08:44 |
<godog> |
silence mendelevium for today, status unclear T111532 |
[production] |
| 08:30 |
<jynus> |
endinf profiling and executing pt-query-digest on db1043 [ETA:4h] |
[production] |
| 07:52 |
<godog> |
reboot ms-be1010 to pick up disk ordering change |
[production] |
| 04:47 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Mon Sep 14 04:47:58 UTC 2015 (duration 47m 57s) |
[production] |
| 02:29 |
<l10nupdate@tin> |
LocalisationUpdate completed (1.26wmf22) at 2015-09-14 02:29:48+00:00 |
[production] |
| 02:26 |
<l10nupdate@tin> |
Synchronized php-1.26wmf22/cache/l10n: l10nupdate for 1.26wmf22 (duration: 06m 59s) |
[production] |
| 01:31 |
<Krinkle> |
mwscript deleteEqualMessages.php --wiki sqwiki |
[production] |
|
2015-09-11
§
|
| 21:21 |
<hashar> |
shutdown nodepool on labnodepool1001.eqiad.wmnet until monday |
[production] |
| 18:01 |
<legoktm@tin> |
Synchronized php-1.26wmf22/extensions/Echo/: Echo regression fixes #2 (duration: 00m 12s) |
[production] |
| 16:43 |
<krinkle@tin> |
Synchronized php-1.26wmf22/resources/src/mediawiki/mediawiki.js: T112232 (duration: 00m 12s) |
[production] |
| 16:37 |
<legoktm@tin> |
Synchronized php-1.26wmf22/extensions/Echo/: Echo regression backports (duration: 00m 12s) |
[production] |
| 16:35 |
<legoktm@tin> |
Synchronized php-1.26wmf22/resources/src/mediawiki/mediawiki.js: resourceloader: Document internal mw.loader#jobs property (again) (duration: 00m 13s) |
[production] |
| 16:33 |
<legoktm> |
ssh: connect to host mw1156.eqiad.wmnet port 22: Connection timed out |
[production] |
| 16:32 |
<paravoid> |
powercycling mw1156, multiple kernel backtraces in console output |
[production] |
| 16:32 |
<legoktm@tin> |
Synchronized php-1.26wmf22/resources/src/mediawiki/mediawiki.js: resourceloader: Document internal mw.loader#jobs property (duration: 01m 07s) |
[production] |
| 16:15 |
<cmjohnson1> |
mw1031 rebooting for f/w update |
[production] |
| 16:07 |
<bblack> |
enabled LRO+GRO on lvs200[123], starting pybal there again ([456] testing looks good so far) |
[production] |
| 15:44 |
<bblack> |
enabled LRO+GRO on lvs200[456] (backups). Stopping pybal on lvs200[123] to test... |
[production] |
| 15:11 |
<cmjohnson1> |
swapping pem2 cr2-eqiad |
[production] |
| 10:03 |
<jynus> |
starting nodepool in labnodepool1001 |
[production] |