|
2014-05-19
§
|
| 14:52 |
<_joe_> |
powercycled mw1171, dead and serial console stuck |
[production] |
| 14:51 |
<manybubbles> |
synchronized cirrus.dblist 'Switch cirrus to the primary backend for zh-yue wikipedia - resyncing to mw1171' |
[production] |
| 14:40 |
<manybubbles> |
synchronized wmf-config/InitialiseSettings.php 'touched and synced InitializeSettings.php to make update to cirrus.dblist take hold' |
[production] |
| 14:38 |
<manybubbles> |
synchronized cirrus.dblist 'Switch cirrus to the primary backend for zh-yue wikipedia' |
[production] |
| 13:34 |
<Krinkle> |
Running deleteEqualMessages.php on mtwiki (bug 43917) |
[production] |
| 13:17 |
<Krinkle> |
Running deleteEqualMessages.php on zh_min_nanwiki (bug 43917) |
[production] |
| 13:07 |
<Krinkle> |
Running deleteEqualMessages.php on zh_yuewiki (bug 43917) |
[production] |
| 12:01 |
<Krinkle> |
Running deleteEqualMessages.php on suwiki (bug 43917) |
[production] |
| 03:10 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Mon May 19 03:09:00 UTC 2014 (duration 8m 59s) |
[production] |
| 02:26 |
<LocalisationUpdate> |
completed (1.24wmf5) at 2014-05-19 02:25:51+00:00 |
[production] |
| 02:14 |
<LocalisationUpdate> |
completed (1.24wmf4) at 2014-05-19 02:13:37+00:00 |
[production] |
| 00:30 |
<Tim> |
on osmium: stopping job runners in order to fix cgroup permissions issue |
[production] |
|
2014-05-17
§
|
| 03:08 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Sat May 17 03:07:12 UTC 2014 (duration 7m 11s) |
[production] |
| 02:25 |
<LocalisationUpdate> |
completed (1.24wmf5) at 2014-05-17 02:24:46+00:00 |
[production] |
| 02:15 |
<LocalisationUpdate> |
completed (1.24wmf4) at 2014-05-17 02:14:13+00:00 |
[production] |
| 01:58 |
<mutante> |
powercycling labsdb1003 |
[production] |
| 00:23 |
<ori> |
varnishadm on cp1056 confirms that varnish recognizes mw1151 as "sick" |
[production] |
| 00:20 |
<ori> |
stopping apache and disabling puppet on mw1151 so that varnish stops forwarding reqs to it |
[production] |
| 00:16 |
<Krinkle> |
On mw1151, Gadget::loadStructuredList() returns false, memcached has no value for 'enwiki:gadgets-definition:7' and is unable to store it. |
[production] |
| 00:11 |
<ori> |
timo identified weird RL responses as all originating in mw1151; dmesg shows ata1 disk troubles: "failed command: READ DMA EXT", "sd 0:0:0:0: [sda] Add. Sense: Unrecovered read error - auto reallocate failed" |
[production] |
| 00:04 |
<krinkle> |
synchronized php-1.24wmf4/includes/resourceloader/ResourceLoader.php 'I718fcf23d' |
[production] |
|
2014-05-16
§
|
| 23:07 |
<awight> |
updated crm from 0b8aa8aa046935b6cfc67c10ebe10396d5e42745 to 65a433b5564f42c3aa4f310cd4bb938ae70f841d |
[production] |
| 22:56 |
<aaron> |
synchronized php-1.24wmf4/includes/db/Database.php '182e42c173b9ab0c2bc5d753879a000b1ff39e77' |
[production] |
| 22:54 |
<aaron> |
synchronized php-1.24wmf5/includes/db/Database.php '8829ffc72d3332d348a1a2e58d525e54e126bad5' |
[production] |
| 22:17 |
<awight> |
tools updated from 85bb7293d83517086e3609f03365aecde9f58c71 to ee31fc94b17c11a48ddac19aabfcdaab69fd2f72 |
[production] |
| 21:37 |
<ori> |
synchronized php-1.24wmf4/extensions/MultimediaViewer 'Update MultimediaViewer for I0df067a61: Add sampling to unsampled event logging' |
[production] |
| 21:33 |
<ori> |
synchronized php-1.24wmf5/extensions/MultimediaViewer 'Update MultimediaViewer for I0df067a61: Add sampling to unsampled event logging' |
[production] |
| 21:10 |
<mwalker> |
updating fundraising smashpig from 2fdf982b20f1cbeaf9f57af64ef21b5b69a36f6e to f64f79f13cf4ab560d0bb5bd69690c827a821629 |
[production] |
| 20:41 |
<awight> |
update crm from 243641de631b712c4a29ca1f3618771b78dadeae to 0b8aa8aa046935b6cfc67c10ebe10396d5e42745 |
[production] |
| 18:43 |
<awight> |
update tools from a40c0caa18a0efd93bc5d3f7f68386fbc36bf1fa to 85bb7293d83517086e3609f03365aecde9f58c71 |
[production] |
| 18:12 |
<ori> |
synchronized wmf-config/InitialiseSettings.php 'I3c453b0949f4e: Tweak MediaViewer sampling settings' |
[production] |
| 18:07 |
<ori> |
synchronized wmf-config 'Ia43821231: Add sampling control setting for MediaViewer event' |
[production] |
| 18:04 |
<ori> |
updated /a/common to {{Gerrit|Ia43821231}}: Add sampling control setting for MediaViewer event logging |
[production] |
| 17:47 |
<demon> |
synchronized wmf-config/CommonSettings.php 'GeoData to Elastic for all wikivoyages' |
[production] |
| 17:11 |
<mwalker> |
updating staging payments servers as well from 5e24b953dcff5305099e152139e6e93daba8aeec to d99177518b741e7fe18ffda86c83f93c72e164a6 |
[production] |
| 17:10 |
<mwalker> |
and updated to 1.22.6 |
[production] |
| 17:10 |
<mwalker> |
moved fundraising wiki from 6a1d4983319038edeb88dc34a1c220ecaec1cbde to d99177518b741e7fe18ffda86c83f93c72e164a6 -- including json i18n changes |
[production] |
| 16:55 |
<manybubbles> |
"in place" reindexing (for cirrus) all the wikipedias after the deploy train hit them yesterday |
[production] |
| 16:53 |
<demon> |
synchronized wmf-config/CommonSettings.php 'Removing old WikiEditor settings' |
[production] |
| 16:44 |
<RobH> |
partial zirconium downtime |
[production] |
| 16:44 |
<RobH> |
i logged into zirconium, but it had recovered by the time I checked it. |
[production] |
| 16:33 |
<mwalker> |
updated fundraising civicrm from 7a23465e620211739421cce3ad57c62597eb8cc3 to 75c1a50b8aa7e7b6f218d7c420932a8fc53a0a34 for an exchange rates fix |
[production] |
| 16:26 |
<qchris> |
updated gerrit's hooks-bugzilla plugin to version 2.8.1.2 to allow talking to bugzilla-4.4.4 |
[production] |
| 13:54 |
<springle> |
synchronized wmf-config/db-eqiad.php 'repool db1056 in s4, warm up' |
[production] |
| 13:17 |
<springle> |
synchronized wmf-config/db-eqiad.php 'raise db1070 and db1071 to normal load' |
[production] |
| 10:14 |
<springle> |
xtrabackup clone db1049 to db1056 |
[production] |
| 10:13 |
<springle> |
synchronized wmf-config/db-eqiad.php 'reduce db1049 load while cloning' |
[production] |