251-300 of 10000 results (20ms)
2013-04-11 §
13:08 <mark> Inserted python-extras packages into the precise-wikimedia APT repository (backport by hashar) [production]
10:10 <hashar> Jenkins is back up [production]
09:23 <hashar> restarting Jenkins to get the git verbose mode :( [production]
08:44 <hashar> jenkins: stop git cloning mw/core over and over by reusing existing clone {{gerrit|58502}}. That should help with I/O starvation on gallium. [production]
08:40 <Nikerabbit> translation memory database rebuild finished [production]
08:17 <Nikerabbit> emptying Translate solr instance and starting rebuilds [production]
08:15 <nikerabbit> synchronized php-1.22wmf1/extensions/Translate/ 'to master' [production]
07:11 <hashar> synchronized docroot/noc/conf/highlight.php 'Revert highlight.php recent modifications {{bug|47112}} {{gerrit|58664}}' [production]
07:01 <hashar> Put a live hack on fenari for {{bug|47112}} [production]
02:16 <LocalisationUpdate> completed (1.22wmf1) at Thu Apr 11 02:16:35 UTC 2013 [production]
01:10 <Ryan_Lane> disabling puppet on labstore3/4 - making sure gluster won't die without them and we're going to rebuild them [production]
2013-04-10 §
23:30 <LeslieCarr> deactivated XO ipv6 transit [production]
22:25 <Ryan_Lane> added live hack to OpenStackManager on wikitech, ensuring that DynamicSidebar support is only enabled for logged-in users [production]
22:23 <aaron> synchronized wmf-config/jobqueue-eqiad.php 'moved refreshLinks to redis.' [production]
22:20 <^demon|sick> restarting gerrit, was hung [production]
22:15 <Ryan_Lane> enable DynamicSidebar support in OpenStackManager [production]
22:14 <Ryan_Lane> temporarily disabling DynamicSidebar support in OpenStackManager [production]
21:59 <notpeter> gracefulling apache on stafford [production]
21:35 <aaron> synchronized wmf-config/jobqueue-eqiad.php 'Moved htmlCacheUpdate jobs to redis' [production]
21:18 <ottomata> added python-statsd to apt repo for hashar [production]
21:18 <aaron> synchronized wmf-config/jobqueue-eqiad.php [production]
21:06 <ottomata> added python-voluptuous to apt repo (for hashar) [production]
20:46 <ottomata> added python-jsonschema to wikimedia apt repo: ori-l :) [production]
20:00 <aaron> synchronized wmf-config/jobqueue-eqiad.php 'Moved async upload jobs to redis' [production]
19:12 <notpeter> running puppetd --enable on all nodes via salt [production]
19:09 <mwalker> eqiad frack is now using clustered memcache and has site local configuration for DB, ActiveMQ, MemCacheD shtuff [production]
18:23 <aaron> synchronized wmf-config/jobqueue-eqiad.php 'Enabled use of redis for null test jobs' [production]
18:06 <aaron> rebuilt wikiversions.cdb and synchronized wikiversions files: Switched all remaning wikis to 1.22wmf1 [production]
17:45 <RobH> authdns-update to rename db1012 to stat1002 [production]
17:28 <aaron> synchronized php-1.22wmf1/maintenance/copyJobQueue.php 'deployed 54f74111901ddebdc69f26847e3904140b3723d5' [production]
17:23 <LeslieCarr> restart unresponsive cp1037 [production]
17:14 <RobH> authdns-update [production]
15:47 <mark> Ran dist-upgrade (for varnish upgrade) on cp1021-1036 [production]
15:40 <demon> synchronized wmf-config/CommonSettings.php 'Nostalgia ext' [production]
15:40 <demon> synchronized wmf-config/extension-list 'Nostalgia ext' [production]
15:38 <demon> synchronized php-1.22wmf1/extensions/Nostalgia [production]
14:33 <maxsem> synchronized wmf-config/mobile.php [production]
14:18 <maxsem> synchronized wmf-config 'Enable $wgMFVaryResources on test' [production]
14:12 <mark> Inserted varnish 3.0.3plus-rc1-wm10 packages into the APT repository [production]
12:22 <apergos> removed outright puppet.log on neon, was > 7gb and no room left on device [production]
07:53 <krinkle> synchronized php-1.22wmf1/resources/startup.js 'Ia54dd738b3ce0995fa' [production]
05:57 <Tim> on stafford: swapdeath repeat narrowly averted via killall ruby && /etc/init.d/apache2 stop [production]
05:34 <Tim> on brewster: proxy down due to full root partition, again. Will fix it properly this time. [production]
05:27 <olivneh> synchronized wmf-config/InitialiseSettings.php 'Add 'Zero' & 'Zero talk' namespaces to metawiki (If6b3ce5e4)' [production]
04:52 <Tim> stafford went into swapdeath. I killed all ruby processes a couple of times, and then eventually stopped apache2 for a while, while it recovered [production]
02:28 <LocalisationUpdate> completed (1.21wmf12) at Wed Apr 10 02:28:06 UTC 2013 [production]
02:17 <LocalisationUpdate> completed (1.22wmf1) at Wed Apr 10 02:17:19 UTC 2013 [production]
00:21 <mutante> removing xenon and db1012 from icinga configs, running puppetstoredconfigclean.rb on them, restarting icinga [production]
00:07 <maxsem> synchronized php-1.21wmf12/extensions/MobileFrontend/includes/MobileFrontend.hooks.php 'https://gerrit.wikimedia.org/r/#/c/58436/' [production]
00:06 <maxsem> synchronized php-1.22wmf1/extensions/MobileFrontend/includes/MobileFrontend.hooks.php 'https://gerrit.wikimedia.org/r/#/c/58436/' [production]