851-900 of 10000 results (23ms)
2015-01-27 §
19:15 <Krinkle> irc.wikimedia.org is down. "Connection refused." [production]
19:14 <Krenair> IRC RC seems broken [production]
18:31 <YuviPanda> rebooting tungstun [production]
18:27 <godog> reboot swift in esams [production]
17:59 <YuviPanda> rebooting labmon1001 [production]
17:49 <godog> reboot all swift machines in eqiad, in turn [production]
17:47 <bblack> rebooting various LVSes... [production]
17:23 <marktraceur> I am consciously leaving NavigationTiming unsynced because nobody seems that concerned about it, and nobody is here to shepherd the patch. If you *are* concerned about it, then contact ori. [production]
17:22 <akosiaris> rebooting sca1001, sca1002, chromium, oxygen [production]
17:21 <marktraceur> Synchronized php-1.25wmf14/extensions/GWToolset/: [SWAT] [wmf14] GWToolset HHVM fixes (duration: 00m 06s) [production]
17:11 <marktraceur> Synchronized php-1.25wmf14/extensions/GWToolset/includes/: [SWAT] [wmf14] GWToolset HHVM fixes (duration: 00m 07s) [production]
17:08 <marktraceur> Synchronized wmf-config/InitialiseSettings.php: [SWAT] [config] Rename portal namespace in kowiki (duration: 00m 05s) [production]
16:54 <marktraceur> Synchronized php-1.25wmf15/extensions/GWToolset/includes/: [SWAT] [wmf15] GWToolset HHVM fixes (duration: 00m 06s) [production]
04:08 <LocalisationUpdate> ResourceLoader cache refresh completed at Tue Jan 27 04:08:16 UTC 2015 (duration 8m 15s) [production]
02:19 <LocalisationUpdate> completed (1.25wmf15) at 2015-01-27 02:19:11+00:00 [production]
02:19 <l10nupdate> Synchronized php-1.25wmf15/cache/l10n: (no message) (duration: 00m 03s) [production]
02:11 <LocalisationUpdate> completed (1.25wmf14) at 2015-01-27 02:10:49+00:00 [production]
02:10 <l10nupdate> Synchronized php-1.25wmf14/cache/l10n: (no message) (duration: 00m 02s) [production]
00:58 <reedy> Synchronized wmf-config/: Noop for citoid for beta (duration: 00m 05s) [production]
00:50 <reedy> Synchronized wmf-config/: Noop for bouncehandler for beta (duration: 00m 06s) [production]
2015-01-26 §
23:57 <_joe_> depooling mw1018 for testing with user changing [production]
20:00 <akosiaris> restarted apache on palladium/strontium, cleared the, created on Jan 23, pid files from puppetmaster [production]
19:59 <_joe_> restarted apache on palladium [production]
19:31 <YuviPanda> restarted keystone on virt1000, ^d couldn’t log in [production]
19:15 <reedy> Synchronized wmf-config/interwiki.cdb: Updating interwiki cache (duration: 00m 06s) [production]
04:08 <LocalisationUpdate> ResourceLoader cache refresh completed at Mon Jan 26 04:08:01 UTC 2015 (duration 8m 0s) [production]
02:23 <LocalisationUpdate> completed (1.25wmf15) at 2015-01-26 02:23:10+00:00 [production]
02:23 <l10nupdate> Synchronized php-1.25wmf15/cache/l10n: (no message) (duration: 00m 03s) [production]
02:15 <LocalisationUpdate> completed (1.25wmf14) at 2015-01-26 02:14:49+00:00 [production]
02:14 <l10nupdate> Synchronized php-1.25wmf14/cache/l10n: (no message) (duration: 00m 02s) [production]
2015-01-25 §
20:50 <ori> depooled mw1118 while investigating T85428 [production]
18:55 <bd808> high rate of nutcracker "SYSTEM ERROR" errors on mw1118 [production]
18:44 <bd808> trimmed Logstash redis input queues to 0 events; dropped ~4M backlogged events [production]
04:14 <LocalisationUpdate> ResourceLoader cache refresh completed at Sun Jan 25 04:14:33 UTC 2015 (duration 14m 32s) [production]
02:23 <LocalisationUpdate> completed (1.25wmf15) at 2015-01-25 02:22:58+00:00 [production]
02:23 <l10nupdate> Synchronized php-1.25wmf15/cache/l10n: (no message) (duration: 00m 02s) [production]
02:10 <LocalisationUpdate> completed (1.25wmf14) at 2015-01-25 02:10:30+00:00 [production]
02:10 <l10nupdate> Synchronized php-1.25wmf14/cache/l10n: (no message) (duration: 00m 02s) [production]
2015-01-24 §
22:40 <bd808> Emptied logstash redis lists on all 3 hosts [production]
22:27 <bd808> Full restart of logstash elasticsearch cluster [production]
06:27 <paravoid> mass-restarting hhvm across the cluster [production]
06:22 <faidon> Synchronized wmf-config: touched config (duration: 00m 07s) [production]
06:04 <faidon> Synchronized wmf-config/StartProfiler.php: fix for T87497/r186578 (duration: 00m 06s) [production]
04:52 <LocalisationUpdate> ResourceLoader cache refresh completed at Sat Jan 24 04:52:42 UTC 2015 (duration 52m 41s) [production]
02:30 <LocalisationUpdate> completed (1.25wmf15) at 2015-01-24 02:30:10+00:00 [production]
02:30 <l10nupdate> Synchronized php-1.25wmf15/cache/l10n: (no message) (duration: 00m 02s) [production]
02:17 <LocalisationUpdate> completed (1.25wmf14) at 2015-01-24 02:17:08+00:00 [production]
02:17 <l10nupdate> Synchronized php-1.25wmf14/cache/l10n: (no message) (duration: 00m 02s) [production]
01:06 <hashar> restarting Jenkins (deadlock on deployment-bastion slave) [production]
00:50 <reedy> Synchronized wmf-config/: Kill wmgVectorBetaPersonalBar (duration: 00m 08s) [production]