1451-1500 of 10000 results (20ms)
2014-12-01 §
13:46 <_joe_> removing the same files from ocg1002,3 as well [production]
13:44 <_joe_> removing cache files from ocg1001, when they're older than 3 days [production]
09:55 <_joe_> reimaging mw1033-mw1040 to HHVM, depooling from the main pool now [production]
09:31 <_joe_> upgrading hhvm to the latest version across the cluster [production]
04:46 <tstarling> Synchronized php-1.25wmf10/includes/parser/MWTidy.php: change previously pulled but scap was apparently not run (duration: 00m 05s) [production]
04:44 <tstarling> Synchronized php-1.25wmf9/includes/parser/MWTidy.php: change previously pulled but scap was apparently not run (duration: 00m 06s) [production]
03:34 <LocalisationUpdate> ResourceLoader cache refresh completed at Mon Dec 1 03:34:57 UTC 2014 (duration 34m 56s) [production]
02:17 <LocalisationUpdate> completed (1.25wmf10) at 2014-12-01 02:17:56+00:00 [production]
02:17 <l10nupdate> Synchronized php-1.25wmf10/cache/l10n: (no message) (duration: 00m 03s) [production]
02:10 <LocalisationUpdate> completed (1.25wmf9) at 2014-12-01 02:10:33+00:00 [production]
02:10 <l10nupdate> Synchronized php-1.25wmf9/cache/l10n: (no message) (duration: 00m 01s) [production]
00:51 <tstarling> Synchronized wmf-config/StartProfiler.php: (no message) (duration: 00m 05s) [production]
2014-11-30 §
23:17 <qchris> Updated EventLogging to 19c23698bc03694017d764af33307d6f035fc224 (on 22:51) and restarted it [production]
20:51 <qchris> restarted eventlogging mysql-m2-master consumer. It seems it could no longer write to the database. [production]
19:17 <Krinkle> Disabling and relauching Gearman connection from Jenkins. [production]
10:28 <oblivian> Synchronized wmf-config/jobqueue-eqiad.php: reverting to rdb1001 (duration: 00m 05s) [production]
10:13 <mark> Rebooted asw-c4-eqiad [production]
09:13 <_joe_> jobsqueues work again [production]
09:08 <oblivian> Synchronized wmf-config/jobqueue-eqiad.php: changing the aggregator address as well (duration: 00m 05s) [production]
07:27 <_joe_> restarted the jobrunner service on all jobrunners [production]
07:15 <oblivian> Synchronized wmf-config/jobqueue-eqiad.php: (no message) (duration: 00m 05s) [production]
05:50 <ori> 3:50 UTC: switch asw-c-eqiad lost connectivity with cabinet C4. Impact: phabricator down; gap in web request logs and some perf monitoring. Job queue and Recent Changes stream OK b/c redundant servers are up. [production]
03:41 <LocalisationUpdate> ResourceLoader cache refresh completed at Sun Nov 30 03:41:03 UTC 2014 (duration 41m 2s) [production]
02:18 <LocalisationUpdate> completed (1.25wmf10) at 2014-11-30 02:18:22+00:00 [production]
02:18 <l10nupdate> Synchronized php-1.25wmf10/cache/l10n: (no message) (duration: 00m 02s) [production]
02:11 <LocalisationUpdate> completed (1.25wmf9) at 2014-11-30 02:11:01+00:00 [production]
02:11 <l10nupdate> Synchronized php-1.25wmf9/cache/l10n: (no message) (duration: 00m 01s) [production]
2014-11-29 §
04:17 <LocalisationUpdate> ResourceLoader cache refresh completed at Sat Nov 29 04:17:41 UTC 2014 (duration 17m 40s) [production]
02:25 <LocalisationUpdate> completed (1.25wmf10) at 2014-11-29 02:25:53+00:00 [production]
02:25 <l10nupdate> Synchronized php-1.25wmf10/cache/l10n: (no message) (duration: 00m 01s) [production]
02:13 <LocalisationUpdate> completed (1.25wmf9) at 2014-11-29 02:13:35+00:00 [production]
02:13 <l10nupdate> Synchronized php-1.25wmf9/cache/l10n: (no message) (duration: 00m 01s) [production]
2014-11-28 §
13:19 <YuviPanda> restarted apache on palladium, things are recovering [production]
11:20 <qchris> Updated gerrit plugin its-phabricator-from-bugzilla to 97c5f02d3ca6259488a763515251c5cc57a11a51 [production]
11:20 <qchris> Updated gerrit plugin its-phabricator to 9edf90a182e43bfeea7ebbcb20d4a52b6213600d [production]
03:39 <LocalisationUpdate> ResourceLoader cache refresh completed at Fri Nov 28 03:39:52 UTC 2014 (duration 39m 51s) [production]
02:22 <LocalisationUpdate> completed (1.25wmf10) at 2014-11-28 02:22:28+00:00 [production]
02:22 <l10nupdate> Synchronized php-1.25wmf10/cache/l10n: (no message) (duration: 00m 04s) [production]
02:10 <LocalisationUpdate> completed (1.25wmf9) at 2014-11-28 02:10:06+00:00 [production]
02:10 <l10nupdate> Synchronized php-1.25wmf9/cache/l10n: (no message) (duration: 00m 01s) [production]
2014-11-27 §
17:55 <_joe_> restarted hhvm on mw1224, the alarm may have been lost in the puppet failure shower earlier [production]
17:24 <godog> removed /var/lib/carbon/whisper/archived/jenkins from tungsten [production]
17:09 <godog> upload txstatsd 0.7.0~bzr30-0ubuntu0+14 to precise-wikimedia on carbon [production]
16:49 <godog> upload missing txstatsd 1.0.0-1 _source package_ to carbon [production]
16:48 <godog> upload missing txstatsd 1.0.0-1 to carbon [production]
15:48 <hoo> Synchronized php-1.25wmf10/extensions/Wikidata/: Fixing a data model bug + enable Statements on Properties for testwikidata (duration: 00m 12s) [production]
15:34 <hoo> Synchronized wmf-config/Wikibase.php: Set "displayStatementsOnProperties" for wikidata/testwikidata (duration: 00m 06s) [production]
14:48 <akosiaris> upgrading librsvg throughout the fleet [production]
04:35 <springle> restarted squid3 on carbon, but glitches seem to be upstream [production]
04:23 <LocalisationUpdate> ResourceLoader cache refresh completed at Thu Nov 27 04:23:45 UTC 2014 (duration 23m 44s) [production]