251-300 of 10000 results (24ms)
2013-04-10 §
19:12 <notpeter> running puppetd --enable on all nodes via salt [production]
19:09 <mwalker> eqiad frack is now using clustered memcache and has site local configuration for DB, ActiveMQ, MemCacheD shtuff [production]
18:23 <aaron> synchronized wmf-config/jobqueue-eqiad.php 'Enabled use of redis for null test jobs' [production]
18:06 <aaron> rebuilt wikiversions.cdb and synchronized wikiversions files: Switched all remaning wikis to 1.22wmf1 [production]
17:45 <RobH> authdns-update to rename db1012 to stat1002 [production]
17:28 <aaron> synchronized php-1.22wmf1/maintenance/copyJobQueue.php 'deployed 54f74111901ddebdc69f26847e3904140b3723d5' [production]
17:23 <LeslieCarr> restart unresponsive cp1037 [production]
17:14 <RobH> authdns-update [production]
15:47 <mark> Ran dist-upgrade (for varnish upgrade) on cp1021-1036 [production]
15:40 <demon> synchronized wmf-config/CommonSettings.php 'Nostalgia ext' [production]
15:40 <demon> synchronized wmf-config/extension-list 'Nostalgia ext' [production]
15:38 <demon> synchronized php-1.22wmf1/extensions/Nostalgia [production]
14:33 <maxsem> synchronized wmf-config/mobile.php [production]
14:18 <maxsem> synchronized wmf-config 'Enable $wgMFVaryResources on test' [production]
14:12 <mark> Inserted varnish 3.0.3plus-rc1-wm10 packages into the APT repository [production]
12:22 <apergos> removed outright puppet.log on neon, was > 7gb and no room left on device [production]
07:53 <krinkle> synchronized php-1.22wmf1/resources/startup.js 'Ia54dd738b3ce0995fa' [production]
05:57 <Tim> on stafford: swapdeath repeat narrowly averted via killall ruby && /etc/init.d/apache2 stop [production]
05:34 <Tim> on brewster: proxy down due to full root partition, again. Will fix it properly this time. [production]
05:27 <olivneh> synchronized wmf-config/InitialiseSettings.php 'Add 'Zero' & 'Zero talk' namespaces to metawiki (If6b3ce5e4)' [production]
04:52 <Tim> stafford went into swapdeath. I killed all ruby processes a couple of times, and then eventually stopped apache2 for a while, while it recovered [production]
02:28 <LocalisationUpdate> completed (1.21wmf12) at Wed Apr 10 02:28:06 UTC 2013 [production]
02:17 <LocalisationUpdate> completed (1.22wmf1) at Wed Apr 10 02:17:19 UTC 2013 [production]
00:21 <mutante> removing xenon and db1012 from icinga configs, running puppetstoredconfigclean.rb on them, restarting icinga [production]
00:07 <maxsem> synchronized php-1.21wmf12/extensions/MobileFrontend/includes/MobileFrontend.hooks.php 'https://gerrit.wikimedia.org/r/#/c/58436/' [production]
00:06 <maxsem> synchronized php-1.22wmf1/extensions/MobileFrontend/includes/MobileFrontend.hooks.php 'https://gerrit.wikimedia.org/r/#/c/58436/' [production]
2013-04-09 §
23:29 <tstarling> synchronized php-1.22wmf1/extensions/Nostalgia [production]
23:21 <notpeter> running upgrades on db56 and rebooting [production]
23:19 <notpeter> running upgrades on db64 and rebooting [production]
22:55 <maxsem> synchronized php-1.21wmf12/extensions/MobileFrontend/ [production]
22:52 <maxsem> synchronized php-1.22wmf1/extensions/MobileFrontend/ [production]
22:36 <reedy> synchronized wmf-config/ [production]
22:33 <LeslieCarr> clearing a few varnish caches - fyi if db load is seen to increase [production]
22:16 <maxsem> Finished syncing Wikimedia installation... : Weekly mobile deployment [production]
21:45 <maxsem> Started syncing Wikimedia installation... : Weekly mobile deployment [production]
21:44 <hashar> Zuul is slowly dequeuing a large number of jobs. There is currently rougly 30 minutes lag between change submission and actually processing of the job. No idea why but investigating. [production]
19:56 <Ryan_Lane> installed Nuke on wikitech [production]
18:11 <K4-713> updated payments anti-fraud rules [production]
13:07 <hashar> jenkins plugin upgrade is tracked in {{bug|47040}}. Luckily git plugin has been fixed just in time ;-D Just need to migrate the config history. [production]
12:37 <hashar> jenkins plugins seems to have been upgraded overnight :( [production]
10:26 <mark> Deployed experimental varnish build (3.0.3plus~rc1-wm10) on dysprosium and cp1022 [production]
08:53 <hashar> Jenkins tests for mediawiki/core are broken ({{bug|47031}}) due to a faulty change that has been merged in master branch. Root cause is the gating job not testing the proper change {{bug| 46723}} :/ [production]
08:43 <Nikerabbit> Got rid of local changes and undeployed commit related to noc [production]
08:41 <nikerabbit> synchronized wmf-config/InitialiseSettings.php 'Narayam and WebFonts' [production]
05:41 <Tim> on mw1149: enabled apache access log [production]
02:31 <LocalisationUpdate> completed (1.21wmf12) at Tue Apr 9 02:30:59 UTC 2013 [production]
02:19 <LocalisationUpdate> completed (1.22wmf1) at Tue Apr 9 02:18:53 UTC 2013 [production]
00:26 <^demon> jenkins is back. java gc took awhile. [production]
00:19 <^demon> jenkins is hung, looking into it. [production]
00:16 <awight> payments cluster updated from 22943dc28 to d3954768e4dd [production]