5001-5050 of 10000 results (20ms)
2014-04-21 §
11:51 <reedy> synchronized php-1.23wmf22/extensions/TimedMediaHandler 'I7483c8b7ec75f5149998da2b530ca04' [production]
11:51 <paravoid> deactivating esams<->HE peering, >90% packet loss between lon<->nyc [production]
11:49 <paravoid> deactivating eqiad<->HE peering, >90% packet loss between lon<->nyc [production]
11:45 <reedy> synchronized php-1.23wmf22/extensions/TimedMediaHandler 'I7483c8b7ec75f5149998da2b530ca0467ac70de7' [production]
03:55 <springle> reset pc100* slaves previously replicating from pmtpa [production]
03:33 <ori> 5.5k fatals over last 20 hrs, of which 3.5k are calls to doTransform() on a non-object at TimedMediaThumbnail.php:201, and 0.9k are Lua API OOMs at LuaSandbox/Engine.php:264 [production]
03:30 <LocalisationUpdate> ResourceLoader cache refresh completed at Mon Apr 21 03:30:15 UTC 2014 (duration 30m 14s) [production]
03:26 <ori> ap_busy_workers spike on image scalers eqiad, started ~2:55, subsided around ~3:20 [production]
02:42 <LocalisationUpdate> completed (1.24wmf1) at 2014-04-21 02:42:30+00:00 [production]
02:29 <LocalisationUpdate> completed (1.23wmf22) at 2014-04-21 02:29:49+00:00 [production]
2014-04-20 §
18:52 <ori> restarted grrrit-wm by following instructions on https://wikitech.wikimedia.org/wiki/Grrrit-wm#Restarting_the_bot [production]
11:12 <Nemo_bis> grrrit dead: 10.28 -!- grrrit-wm [tools.lolr@208.80.155.145] has quit [production]
03:26 <LocalisationUpdate> ResourceLoader cache refresh completed at Sun Apr 20 03:26:48 UTC 2014 (duration 26m 47s) [production]
02:39 <LocalisationUpdate> completed (1.24wmf1) at 2014-04-20 02:39:23+00:00 [production]
02:28 <LocalisationUpdate> completed (1.23wmf22) at 2014-04-20 02:28:09+00:00 [production]
2014-04-19 §
03:28 <LocalisationUpdate> ResourceLoader cache refresh completed at Sat Apr 19 03:27:52 UTC 2014 (duration 27m 51s) [production]
02:41 <LocalisationUpdate> completed (1.24wmf1) at 2014-04-19 02:41:02+00:00 [production]
02:29 <LocalisationUpdate> completed (1.23wmf22) at 2014-04-19 02:29:33+00:00 [production]
2014-04-18 §
21:00 <hashar> Jenkins renamed mw-jenkinsbot irc bot to wmf-insecte (french for "bug"). Updated IRC conf to point to chat.freenode.net:7000 with SSL. [production]
19:02 <bblack> enabled cp30[14] varnish mobile frontends in esams pybal [production]
17:50 <bblack> cp301[34] reinstalls complete, should stay ok in monitoring [production]
17:48 <ottomata> resinsalling elastic1008 [production]
16:20 <springle> db48 mysqld shutdown for decom [production]
16:20 <bblack> ignore cp301[34] msgs, reinstalling them [production]
16:10 <springle> db63 mysqld shutdown for decom [production]
15:53 <ottomata> reinstalling elastic1007 [production]
15:52 <springle> db48 mysqld set read_only, disabled m2 repl to db1048 [production]
15:51 <ottomata> disabling puppet on elasti1007 and elastic1008 for reformatting [production]
15:45 <mutante> DNS update - removing Tampa msbe/msfe [production]
15:38 <Jeff_Green> switched mchenry to use db1048/db1049 for OTRS address lookups [production]
15:24 <mutante> DNS update - removing all the Tampa mw/srv mgmt [production]
15:15 <mutante> DNS update - removing lvs1-6 [production]
14:54 <mutante> es5,es6 - revoke puppet certs, salt keys, icinga [production]
14:51 <ottomata> powering down stat1 for decom [production]
14:43 <mutante> ms-fe[14] - shutting down [production]
14:41 <ottomata> disabling puppet on stat1 for decom [production]
14:37 <mutante> ms-be 1-12, Tampa Swift boxes, shutdown [production]
14:24 <mutante> ms-fe[14] - stop puppet,revoke certs,remove icinga [production]
13:54 <mutante> ms-be1-12 - removing from puppet,salt,icinga [production]
13:07 <mutante> Bugzilla Apache, changed SSL cipher suite in I7e9adc182dc ,might cost a a few % performance but zirconium had plenty [production]
11:49 <hashar> removing mw-jenkinsbot (the wikimedia jenkins installation) from #wikimedia-labs [production]
10:10 <hashar> Jenkins upgraded to 1.532.3. [production]
10:06 <hashar> Upgrading Jenkins to latest LTS version 1.532.3 [production]
07:57 <mutante> DNS update - remove api.svc, arptest.pmtpa .. [production]
06:31 <demon> synchronized wmf-config/InitialiseSettings.php 'Next round of wikis done building Cirrus indexes, throw into beta mode' [production]
04:04 <LocalisationUpdate> ResourceLoader cache refresh completed at Fri Apr 18 04:04:21 UTC 2014 (duration 4m 20s) [production]
03:06 <LocalisationUpdate> completed (1.24wmf1) at 2014-04-18 03:06:06+00:00 [production]
02:40 <LocalisationUpdate> completed (1.23wmf22) at 2014-04-18 02:39:51+00:00 [production]
00:21 <ori> synchronized wmf-config/CommonSettings.php 'Ie9b265be9: Enable GlobalCssJs on testwiki & test2wiki (2/2)' [production]
00:21 <ori> synchronized wmf-config/InitialiseSettings.php 'Ie9b265be9: Enable GlobalCssJs on testwiki & test2wiki (1/2)' [production]