|
2014-07-08
§
|
| 15:04 |
<anomie> |
Synchronized php-1.24wmf12/extensions/UploadWizard/UploadWizard.config.php: SWAT: Flickr API is https-only now [[gerrit:144583]] (duration: 00m 10s) |
[production] |
| 13:34 |
<springle> |
slow transaction rollback in progress on db1001 librenms. other databases not affected, but librenms writes are timing out |
[production] |
| 13:32 |
<cmjohnson1> |
replacing disk disk 6 ms-be1005 |
[production] |
| 13:30 |
<cmjohnson1> |
replacing disk 4 ms-be1007 |
[production] |
| 12:38 |
<YuviPanda> |
disregard previous log message, was meant for labs |
[production] |
| 12:37 |
<YuviPanda> |
graphite reduced metrics count from 65k to 25k, monitoring io performance |
[production] |
| 06:57 |
<springle> |
Synchronized wmf-config/db-eqiad.php: raise db traffic samplers to normal load (duration: 00m 06s) |
[production] |
| 05:10 |
<springle> |
Synchronized wmf-config/db-eqiad.php: repool db1010, warm up (duration: 00m 06s) |
[production] |
| 04:51 |
<springle> |
Synchronized wmf-config/db-eqiad.php: depool db1010 for upgrade (duration: 00m 06s) |
[production] |
| 03:27 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Tue Jul 8 03:25:51 UTC 2014 (duration 25m 50s) |
[production] |
| 03:00 |
<LocalisationUpdate> |
completed (1.24wmf12) at 2014-07-08 02:59:33+00:00 |
[production] |
| 02:30 |
<LocalisationUpdate> |
completed (1.24wmf11) at 2014-07-08 02:29:00+00:00 |
[production] |
| 01:17 |
<springle> |
Synchronized wmf-config/db-eqiad.php: repool db1061, warm up (duration: 00m 06s) |
[production] |
| 00:57 |
<springle> |
Synchronized wmf-config/db-eqiad.php: depool db1061 for upgrade (duration: 00m 07s) |
[production] |
| 00:02 |
<^d> |
gerrit upgraded to 2.8.1-4-ga1048ce from 2.8.1-2-g724b796, back up. Might be slow for a bit while caches warm. |
[production] |
|
2014-07-07
§
|
| 23:38 |
<maxsem> |
Synchronized php-1.24wmf12/extensions/ParserFunctions/: https://gerrit.wikimedia.org/r/#q,144510,n,z (duration: 00m 03s) |
[production] |
| 23:38 |
<maxsem> |
Synchronized php-1.24wmf12/includes/StubObject.php: https://gerrit.wikimedia.org/r/#/c/144509/ (duration: 00m 03s) |
[production] |
| 23:22 |
<maxsem> |
Synchronized visualeditor-default.dblist: (no message) (duration: 00m 03s) |
[production] |
| 23:19 |
<maxsem> |
Synchronized php-1.24wmf12/extensions/GWToolset: (no message) (duration: 00m 03s) |
[production] |
| 23:18 |
<maxsem> |
Synchronized php-1.24wmf11/extensions/GWToolset: (no message) (duration: 00m 03s) |
[production] |
| 23:17 |
<maxsem> |
Synchronized php-1.24wmf12/extensions/GWToolset: (no message) (duration: 00m 04s) |
[production] |
| 23:12 |
<maxsem> |
Synchronized wmf-config/: https://gerrit.wikimedia.org/r/#q,144476,n,z & https://gerrit.wikimedia.org/r/#q,139569,n,z (duration: 00m 05s) |
[production] |
| 23:04 |
<maxsem> |
Synchronized wmf-config/: https://gerrit.wikimedia.org/r/#q,144476,n,z & https://gerrit.wikimedia.org/r/#q,139569,n,z (duration: 00m 03s) |
[production] |
| 22:41 |
<ori> |
Synchronized wmf-config/mc.php: I8b66e9339: Make app servers connect to nutcracker on port 11212 (duration: 00m 03s) |
[production] |
| 20:31 |
<ori> |
Synchronized wmf-config/mc.php: Iea24b092b: Make mw1041 connect to nutcracker on port 11212 (duration: 00m 09s) |
[production] |
| 20:03 |
<subbu> |
deployed Parsoid 8ef7b6fe |
[production] |
| 17:52 |
<legoktm> |
deleted rows in centralauth's localnames and localuser tables for bug 67548 |
[production] |
| 17:02 |
<demon> |
Synchronized wmf-config/InitialiseSettings.php: Cirrus on commons as primary (duration: 00m 04s) |
[production] |
| 16:34 |
<aude> |
Finished scap: Update Wikidata to mw1.24-wmf12 branch for group0 wikis (duration: 22m 33s) |
[production] |
| 16:22 |
<manybubbles> |
(Cirrus) load tested commons and eswiki over the last hour - both look fine. |
[production] |
| 16:11 |
<aude> |
Started scap: Update Wikidata to mw1.24-wmf12 branch for group0 wikis |
[production] |
| 15:49 |
<bd808> |
Logstash event volume looks better after restart. Probably related to bug 63490. |
[production] |
| 15:33 |
<bd808> |
Restarted logstash on logstash1001 because log volume looked lower than I though it should be. |
[production] |
| 15:16 |
<cmjohnson1> |
reseating PEM2 cr1-eqiad |
[production] |
| 15:08 |
<godog> |
powercycled ms-be1007, unresponsive on console and remnants of a stack trace |
[production] |
| 14:49 |
<manybubbles> |
(Cirrus) Applying cache warmer configuration that went out last Thursday to all wikipedias. |
[production] |
| 12:11 |
<hashar> |
Jenkins job builder e1ddd23 fails for us :/ Moving back to parent commit |
[production] |
| 12:09 |
<hashar> |
Updated our Jenkins job builder fork 0972985..e1ddd23 |
[production] |
| 09:40 |
<godog> |
upgrade ms-be1003/1004/1012 (zone2) to swift icehouse |
[production] |
| 09:16 |
<_joe_> |
restarting rhenium, pings but no ssh since 2 days, serial console is blank and unresponsive |
[production] |
| 09:15 |
<godog> |
upgrade ms-be1002/1008 (zone1) to swift icehouse |
[production] |
| 02:53 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Mon Jul 7 02:52:10 UTC 2014 (duration 52m 9s) |
[production] |
| 02:24 |
<LocalisationUpdate> |
completed (1.24wmf12) at 2014-07-07 02:23:48+00:00 |
[production] |
| 02:13 |
<LocalisationUpdate> |
completed (1.24wmf11) at 2014-07-07 02:12:44+00:00 |
[production] |