6301-6350 of 10000 results (36ms)
2014-06-23 §
17:04 <bd808> Fixed dangling symlink for /etc/apache2/sites-enabled/logstash.wikimedia.org on logstash1001 by deleting symlink and forcing puppet run [production]
16:49 <godog> added mw1149-52 back to pybal apache [production]
16:33 <paravoid> switched inbound mail for all non-wikimedia.org domains from mchenry/sodium to polonium/mchenry (~16:00 + <= 1h TTL UTC) [production]
15:13 <anomie> Synchronized wmf-config/InitialiseSettings.php: SWAT: Add a Library of Congress domain to wgCopyUploadsDomains [[gerrit:141308]] (duration: 00m 14s) [production]
15:11 <anomie> Synchronized wmf-config/InitialiseSettings.php: SWAT: Adjust group rights on ruwiki [[gerrit:140910]] (duration: 00m 14s) [production]
15:10 <anomie> Synchronized php-1.24wmf9/includes/api/ApiExpandTemplates.php: SWAT: Fix fatal in API action=expandtemplates with Scribunto [[gerrit:141416]] (duration: 00m 15s) [production]
15:04 <anomie> Synchronized php-1.24wmf10/includes/api/ApiExpandTemplates.php: SWAT: Fix fatal in API action=expandtemplates with Scribunto [[gerrit:141417]] (duration: 00m 14s) [production]
14:55 <andrewbogott> reenabling puppet on labstore1001, hoping it doesn't break labs [production]
14:38 <hashar> Further upgraded Zuul up to upstream b8c24ce + our local hacks. Git tag is wmf-deploy-20140623-4 [production]
14:14 <hashar> upgraded Zuul by one commit (that introduces swift supports though disabled it on our setup via a custom hack) [production]
13:20 <paravoid> switching outbound email to polonium [production]
12:17 <manybubbles> rebuilding Cirrus index on group0 wikis to pick up changes like results boosting from categories and wikitext search [production]
10:37 <godog> powering down maerlant, decom-med [production]
10:05 <godog> hardreset maerlant, stuck on console and no ssh [production]
09:40 <paravoid> killing sodium's lighttpd compress cache [production]
07:21 <_joe_> powercycled cp4018, stuck with a blank console [production]
02:59 <springle> moving lighttpd compressed archives on sodium off / to regain inodes [production]
02:46 <LocalisationUpdate> ResourceLoader cache refresh completed at Mon Jun 23 02:45:24 UTC 2014 (duration 45m 23s) [production]
02:27 <LocalisationUpdate> completed (1.24wmf10) at 2014-06-23 02:25:53+00:00 [production]
02:15 <LocalisationUpdate> completed (1.24wmf9) at 2014-06-23 02:13:53+00:00 [production]
00:38 <legoktm> mail is stuck, lots of mails queued in exim [production]
2014-06-22 §
22:25 <_joe_> restarted apache on strontium, passenger crashed (again). [production]
21:06 <hoo> Synchronized wmf-config/InitialiseSettings-labs.php: For cluster consistency... (duration: 00m 08s) [production]
19:24 <godog> silenced LVS healthcheck on rendering.svc until 23:23 UTC [production]
02:42 <LocalisationUpdate> ResourceLoader cache refresh completed at Sun Jun 22 02:41:30 UTC 2014 (duration 41m 29s) [production]
02:24 <LocalisationUpdate> completed (1.24wmf10) at 2014-06-22 02:23:50+00:00 [production]
02:14 <LocalisationUpdate> completed (1.24wmf9) at 2014-06-22 02:12:50+00:00 [production]
2014-06-21 §
16:13 <_joe_> restarted ms-be1012, see http://paste.debian.net/hidden/d3d9bdd7/ for console output (will expire in 3d) [production]
02:46 <LocalisationUpdate> ResourceLoader cache refresh completed at Sat Jun 21 02:45:17 UTC 2014 (duration 45m 16s) [production]
02:30 <LocalisationUpdate> completed (1.24wmf10) at 2014-06-21 02:28:59+00:00 [production]
02:18 <LocalisationUpdate> completed (1.24wmf9) at 2014-06-21 02:17:18+00:00 [production]
2014-06-20 §
22:58 <demon> Synchronized wmf-config/CommonSettings.php: Load nostalgia from skins rather than extensions when it exists (duration: 00m 04s) [production]
20:23 <demon> Synchronized wmf-config/CommonSettings.php: wfGetIP removal, code cleanup (duration: 00m 04s) [production]
20:22 <demon> Synchronized wmf-config/throttle.php: wfGetIP removal, code cleanup (duration: 00m 05s) [production]
17:11 <godog> expanded palladium's root to avoid filling up, suspected salt-master (RT #7721) [production]
16:53 <bd808> Ran /usr/local/bin/sync-common on fenari to verify fix for bug 66844. It works! [production]
15:16 <reedy> Synchronized wmf-config/extension-list-labs: (no message) (duration: 00m 16s) [production]
11:00 <_joe_> restarted apache on palladium, passenger was dead and filling error logs [production]
03:35 <LocalisationUpdate> ResourceLoader cache refresh completed at Fri Jun 20 03:34:06 UTC 2014 (duration 34m 5s) [production]
03:19 <LocalisationUpdate> completed (1.24wmf10) at 2014-06-20 03:18:36+00:00 [production]
02:35 <LocalisationUpdate> completed (1.24wmf9) at 2014-06-20 02:34:14+00:00 [production]
00:06 <MaxSem> Running clearMessageBlobs.php [production]
2014-06-19 §
23:52 <MaxSem> that was a touch [production]
23:51 <maxsem> Synchronized php-1.24wmf9/extensions/MultimediaViewer/: (no message) (duration: 00m 04s) [production]
23:38 <maxsem> Finished scap: Mark Traceur made me do it! (duration: 15m 14s) [production]
23:23 <maxsem> Started scap: Mark Traceur made me do it! [production]
23:20 <maxsem> Synchronized php-1.24wmf10/extensions/MobileFrontend/: (no message) (duration: 00m 03s) [production]
23:20 <maxsem> Synchronized php-1.24wmf9/extensions/MobileFrontend/: (no message) (duration: 00m 04s) [production]
23:18 <maxsem> Synchronized php-1.24wmf10/extensions/CirrusSearch/: (no message) (duration: 00m 03s) [production]
23:18 <maxsem> Synchronized php-1.24wmf10/extensions/VisualEditor/: (no message) (duration: 00m 04s) [production]