| 2014-01-22
      
      § | 
    
  | 20:42 | <RobH> | blog updated | [production] | 
            
  | 19:40 | <matanya> | git.wikimedia.org is down | [production] | 
            
  | 19:32 | <reedy> | synchronized wmf-config/ | [production] | 
            
  | 17:40 | <RobH> | virt0 puppet agent re-enabled and resumed normal service | [production] | 
            
  | 17:05 | <RobH> | issues with change, don't re-enable puppet agent on virt0 yet | [production] | 
            
  | 16:59 | <RobH> | rolling certificate changes to virt1000 and virt0.  virt0 puppet agent disabled for moment | [production] | 
            
  | 16:33 | <Nemo_bis> | still no data in Global_JobQueue_length ganglia graph on hume (3 days now) | [production] | 
            
  | 14:34 | <csteipp> | synchronized php-1.23wmf10/includes/media  'bug60339' | [production] | 
            
  | 14:33 | <csteipp> | synchronized php-1.23wmf11/includes/media  'bug60339' | [production] | 
            
  | 11:18 | <apergos> | powercycled kaulen, it had died the swap death | [production] | 
            
  | 10:02 | <apergos> | restarted mw1206 a little while ago, it was inaccessible even via mgmt | [production] | 
            
  | 06:31 | <reedy> | synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 05:37 | <ori> | restarting hafnium for kernel upgrade | [production] | 
            
  | 03:33 | <LocalisationUpdate> | ResourceLoader cache refresh completed at 2014-01-22 03:33:21+00:00 | [production] | 
            
  | 00:55 | <marktraceur> | deployed config for jsduck jobs for MultimediaViewer | [production] | 
            
  | 00:22 | <demon> | synchronized php-1.23wmf11/extensions/CirrusSearch  'Cirrus to master, with better job queues' | [production] | 
            
  | 00:21 | <demon> | synchronized php-1.23wmf10/extensions/CirrusSearch  'Cirrus to master, with better job queues' | [production] | 
            
  | 00:18 | <demon> | synchronized php-1.23wmf10/extensions/Elastica  'Elastica to master' | [production] | 
            
  | 00:15 | <demon> | synchronized php-1.23wmf10/extensions/CirrusSearch  'Rolling back Cirrus -- fatals' | [production] | 
            
  | 00:11 | <demon> | synchronized php-1.23wmf10/extensions/CirrusSearch  'Cirrus to master, with better job queues' | [production] | 
            
  
    | 2014-01-21
      
      § | 
    
  | 23:55 | <marktraceur> | deployed documentation jenkins jobs for MultimediaViewer | [production] | 
            
  | 23:52 | <awight> | crm updated from e82d2c5e64e67bee9bfa1f703fc94602addea2cb to f71526fa021fe3963a5d94151881f5a8a5c40dab | [production] | 
            
  | 20:24 | <cmjohnson1> | removing iptable rules for loudon and erzurumi | [production] | 
            
  | 20:20 | <awight> | updated listeners from dc21b3bf6560402238f9093f1dfbafe22be0a511 to 5e18193c5bb703e44c5cda22009ca97c295973e3 | [production] | 
            
  | 20:06 | <awight> | updated listeners from 888561ae0ee5a1d5f86da98ab3062f674726717f to dc21b3bf6560402238f9093f1dfbafe22be0a511 | [production] | 
            
  | 19:51 | <awight> | updated listeners from 7c5396c5954e28fe58033008a3b34ebf076a39a5 to 888561ae0ee5a1d5f86da98ab3062f674726717f | [production] | 
            
  | 19:42 | <ori> | synchronized wmf-config/CommonSettings.php  'Ie9ff56755: Add wmgUniversalLanguageSelectorDefault (2/2)' | [production] | 
            
  | 19:42 | <ori> | synchronized wmf-config/InitialiseSettings.php  'Ie9ff56755: Add wmgUniversalLanguageSelectorDefault' | [production] | 
            
  | 19:41 | <ori> | updated /a/common to {{Gerrit|Ie9ff56755}}: Add 'wmgUniversalLanguageSelectorDefault' | [production] | 
            
  | 19:17 | <ori> | synchronized php-1.23wmf10/extensions/UniversalLanguageSelector/Resources.php  'I05c76e478: Make ext.uls.mediawiki depend upon ext.uls.init' | [production] | 
            
  | 19:08 | <ori> | sync-file: mw1206: ssh: connect to host mw1206 port 22: No route to host | [production] | 
            
  | 19:07 | <ori> | synchronized php-1.23wmf11/extensions/UniversalLanguageSelector/Resources.php  'I05c76e478: Make ext.uls.mediawiki depend upon ext.uls.init' | [production] | 
            
  | 18:58 | <awight> | updated SmashPig to 43a27bd379632792bd7fc3ffc26c859d6dbbe0d6 | [production] | 
            
  | 17:06 | <demon> | finished scap: No code updates, just rebuilding all i18n (duration: 58m 29s) | [production] | 
            
  | 16:13 | <demon> | started scap: No code updates, just rebuilding all i18n | [production] | 
            
  | 11:59 | <twkozlowski> | bits application servers back to normal since around 11:46 UTC | [production] | 
            
  | 11:32 | <ori> | bits application server overload due to I71b70d8ee | [production] | 
            
  | 11:25 | <ori> | synchronized php-1.23wmf10/extensions/UniversalLanguageSelector  'I71b70d8ee: Add user preference to enable ULS' | [production] | 
            
  | 11:24 | <ori> | synchronized php-1.23wmf11/extensions/UniversalLanguageSelector  'I71b70d8ee: Add user preference to enable ULS' | [production] | 
            
  | 08:19 | <andrewbogott> | For wikitech labs interface:  deployed new servicegroup schema and supporting code; ran maintenance/transitionServiceGroupSchema.php | [production] | 
            
  | 03:47 | <ori> | synchronized php-1.23wmf10/extensions/TimedMediaHandler/TimedMediaHandler.hooks.php  'Update TimedMediaHandler for I7a6da6c62' | [production] | 
            
  | 03:46 | <ori> | synchronized php-1.23wmf11/extensions/TimedMediaHandler/TimedMediaHandler.hooks.php  'Update TimedMediaHandler for I7a6da6c62' | [production] | 
            
  | 02:50 | <LocalisationUpdate> | ResourceLoader cache refresh completed at 2014-01-21 02:50:24+00:00 | [production] | 
            
  | 02:29 | <ori> | restarting gitblit using new upstart job def (I3f32dedf1) | [production] | 
            
  | 02:28 | <LocalisationUpdate> | completed (1.23wmf11) at 2014-01-21 02:28:40+00:00 | [production] | 
            
  | 02:15 | <LocalisationUpdate> | completed (1.23wmf10) at 2014-01-21 02:15:20+00:00 | [production] |