701-750 of 10000 results (26ms)
2014-11-20 §
16:59 <_joe_> restart apache on mw1218, stuck in a apc futex [production]
16:38 <demon> Synchronized php-1.25wmf9/extensions/Math: (no message) (duration: 00m 06s) [production]
16:26 <_joe_> puppet reenabled everywhere, change tested and live on all varnishes within the next 20 minutes [production]
16:15 <demon> Synchronized php-1.25wmf9/extensions/CirrusSearch: (no message) (duration: 00m 04s) [production]
16:15 <demon> Synchronized php-1.25wmf8/extensions/CirrusSearch: (no message) (duration: 00m 05s) [production]
15:52 <_joe_> disabling puppet on all caches, before a pretty large change, will be reeanbled after a few tests [production]
15:01 <hashar> Restarting Jenkins AND Zuul. Beta cluster jobs are still deadlocked. [production]
14:00 <godog> restart txstatsd on tungsten to stop receiving jenkins metrics [production]
13:04 <hashar> Jenkins: restarting to remove a deadlock and unload the Statsd plugin [production]
12:30 <godog> upload carbon-c-relay to trusty-wikimedia [production]
12:12 <qchris> Restarted EventLogging mysql-m2 consumer to empty its caches [production]
05:41 <bblack> amssq31-62, cp300[12], lvs300[34], ssl300[123] all shut down for esams power event (and downtimed) [production]
05:28 <bblack> ~30m to esams power out, starting equipment shutdown and such for OE13/OE15 [production]
05:15 <Tim> made myself an administrator on phabricator [production]
04:27 <LocalisationUpdate> ResourceLoader cache refresh completed at Thu Nov 20 04:27:25 UTC 2014 (duration 27m 24s) [production]
02:34 <LocalisationUpdate> completed (1.25wmf9) at 2014-11-20 02:34:12+00:00 [production]
02:21 <LocalisationUpdate> completed (1.25wmf8) at 2014-11-20 02:21:35+00:00 [production]
00:50 <maxsem> Synchronized php-1.25wmf9/extensions/MobileFrontend/: https://gerrit.wikimedia.org/r/#/c/174613/ (duration: 00m 04s) [production]
00:41 <catrope> Synchronized wmf-config/InitialiseSettings.php: Change mobile wordmark image to relative URL (duration: 00m 04s) [production]
00:34 <catrope> Synchronized php-1.25wmf9/extensions/VisualEditor: SWAT (duration: 00m 04s) [production]
00:34 <catrope> Synchronized php-1.25wmf8/extensions/VisualEditor: SWAT (duration: 00m 04s) [production]
00:26 <catrope> Synchronized php-1.25wmf8/includes/media/: SWAT: don't apply EXIF rotation to chained thumbnails (duration: 00m 04s) [production]
00:13 <catrope> Synchronized wmf-config/: SWAT: temp debugging for SecurePoll (duration: 00m 04s) [production]
00:10 <catrope> Synchronized wmf-config/: SWAT (duration: 00m 04s) [production]
00:09 <catrope> Synchronized images/mobile/: SWAT: new Wikipedia wordmark for mobile (duration: 00m 03s) [production]
2014-11-19 §
22:01 <ottomata> starting trusty upgrade of analytics1041 [production]
21:38 <yuvipandajs> restarted txstatsd & carbon on labmon1001, recovering from missing points now [production]
21:35 <reedy> Synchronized wmf-config/: ContactPage for legal (duration: 00m 17s) [production]
21:34 <reedy> Synchronized docroot and w: (no message) (duration: 00m 15s) [production]
21:25 <ottomata> starting trusty upgrade of analytics1040 [production]
21:22 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.25wmf9 [production]
21:21 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: Wikipedias to 1.25wmf8 [production]
21:18 <reedy> Finished scap: testwiki to 1.25wmf9 and build l10n cache (duration: 105m 06s) [production]
20:57 <ottomata> starting trusty upgrade of analytics1038 [production]
20:23 <ottomata> starting trusty upgrade of analytics1038 [production]
19:49 <bblack> starting the long slow process of draining out esams traffic ahead of power maint event [production]
19:34 <ottomata> starting trusty upgrade of analytics1037 [production]
19:33 <reedy> Started scap: testwiki to 1.25wmf9 and build l10n cache [production]
19:30 <andrewbogott> upgrading other compute hosts: virt1001-1009 [production]
19:25 <ori> disarming keyholder agent on tin to test alerts [production]
18:53 <reedy> Synchronized php-1.25wmf8/extensions/Wikidata: Ie105a80aa776769eb0dae8a44cda0b7dbe018fb5 (duration: 00m 22s) [production]
18:50 <andrewbogott> upgrading virt1006 [production]
18:36 <andrewbogott> upgrading labnet1001 [production]
18:33 <andrewbogott> upgraded glance on virt1000 to version icehouse [production]
18:29 <ottomata> starting trusty upgrade of 1036 [production]
18:18 <reedy> Synchronized wmf-config/: BounceHandler (duration: 00m 15s) [production]
18:17 <reedy> Synchronized php-1.25wmf8/extensions/BounceHandler/: Bump (duration: 00m 14s) [production]
17:57 <andrewbogott> disabled keystone-redis because the current package doesn't work with icehouse [production]
16:36 <andrewbogott> upgrading virt1000 [production]
16:16 <andrewbogott> moved virt1000 db backup to /a/osback because it was /way/ too big to fit in my homedir [production]