701-750 of 10000 results (18ms)
2013-05-06 §
18:41 <RobH> ns2/nescio appears offline, investigating [production]
18:37 <RobH> ns2 appears offline [production]
18:36 <RobH> authdns-update crashed on ns2, rerunning [production]
18:33 <RobH> holmium [production]
18:33 <RobH> authdns-update for new server homium [production]
18:07 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: enwiki to 1.22wmf3 [production]
17:17 <LeslieCarr> marmontel's interface is now in the sandbox vlan [production]
17:15 <LeslieCarr> turned off marmontel's network port [production]
16:11 <reedy> synchronized docroot [production]
13:54 <hashar> gallium: installed manually libcidr0-dev libanon0-dev . Puppet change is {{gerrit|62434}} [production]
09:51 <paravoid> upgrading Ganglia Web to 3.5.7 [production]
08:25 <hashar> Zuul upgrading to upstream version 7191ee8 + 1 wmf hack. [production]
06:45 <Nemo_bis> Translate search on vanadium keeps being broken https://toolserver.org/~nemobis/tmp/SearchTranslations.log [production]
06:14 <paravoid> pushing new swift rings: ms-be1 weight 50->0, ms-be2 33->75 [production]
02:18 <paravoid> restarting gerrit [production]
02:18 <LocalisationUpdate> failed: git pull of extensions failed [production]
2013-05-05 §
10:56 <Nemo_bis> "Search service is temporarily unavailable" since unknown time https://meta.wikimedia.org/wiki/Special:SearchTranslations?query=amen&uselang=en [production]
03:36 <LocalisationUpdate> ResourceLoader cache refresh completed at Sun May 5 03:36:17 UTC 2013 [production]
02:25 <LocalisationUpdate> completed (1.22wmf2) at Sun May 5 02:25:44 UTC 2013 [production]
02:17 <LocalisationUpdate> completed (1.22wmf3) at Sun May 5 02:17:22 UTC 2013 [production]
2013-05-04 §
22:41 <reedy> synchronized wmf-config/interwiki.cdb 'Updating interwiki cache' [production]
21:44 <Reedy> Manually force ran TorBlock/loadExitNodes.php [production]
21:42 <reedy> synchronized php-1.22wmf3/extensions/TorBlock/ [production]
16:02 <reedy> synchronized wmf-config/ [production]
16:02 <reedy> synchronized docroot/bits/WikipediaMobileFirefoxOS/ [production]
04:36 <LocalisationUpdate> ResourceLoader cache refresh completed at Sat May 4 04:36:09 UTC 2013 [production]
02:20 <LocalisationUpdate> completed (1.22wmf2) at Sat May 4 02:20:30 UTC 2013 [production]
02:12 <LocalisationUpdate> completed (1.22wmf3) at Sat May 4 02:12:01 UTC 2013 [production]
01:17 <Krinkle> binasher restarted gerrit [production]
00:48 <Krinkle> as of an hour ago Gerrit is completely unresponsive. Filed details at https://bugzilla.wikimedia.org/show_bug.cgi?id=48061 [production]
2013-05-03 §
22:19 <py> synchronized wmf-config/db-pmtpa.php 'adding new pmtpa slave' [production]
21:09 <cmjohnson1> db10 is decommissioned...powering down for the final time [production]
20:03 <maxsem> synchronized php-1.22wmf2/extensions/MobileFrontend/includes/MobileFrontend.hooks.php [production]
19:54 <maxsem> synchronized php-1.22wmf3/extensions/MobileFrontend/includes/MobileFrontend.hooks.php [production]
07:51 <MaxSem> OMG logmsgbot is a terminator! [production]
06:16 <tstarling> synchronized wmf-config/missing.php [production]
04:03 <LocalisationUpdate> ResourceLoader cache refresh completed at Fri May 3 04:03:12 UTC 2013 [production]
02:26 <LocalisationUpdate> completed (1.22wmf2) at Fri May 3 02:26:02 UTC 2013 [production]
02:15 <LocalisationUpdate> completed (1.22wmf3) at Fri May 3 02:15:45 UTC 2013 [production]
01:25 <hashar> Zuul processed all its events, seems to be fine now. *sleep* [production]
00:32 <dzahn> synchronized docroot [production]
00:32 <mutante> sync-docroot for iegcom wiki [production]
00:25 <aaron> synchronized multiversion/MWMultiVersion.php '6512451451ac6617bedbfce7be17045bed794d70' [production]
00:15 <mutante> DNS update - add iegcom [production]
2013-05-02 §
23:20 <mutante> restarting lucene on search1021, search1022, search1017, search1018 (with some waiting in between) [production]
23:17 <hashar> yeah after hours and hours of fighting, Jenkins is finally working again. [production]
23:10 <aaron> synchronized wmf-config/jobqueue-eqiad.php [production]
22:58 <hashar> restarting jenkins… it got a few threads blocked and the main process is at 100% usage for now reason [production]
22:58 <mutante> repooling search1016 [production]
22:40 <mutante> gallium, run puppet, graceful Apache to deploy split log files [production]