| 
      
        2016-10-06
      
      ยง
     | 
  
    
  | 22:04 | 
  <thcipriani@tin> | 
  Synchronized php-1.28.0-wmf.21/includes/libs/rdbms/loadbalancer/LoadBalancer.php: [[gerrit:314600|Ignore reuseConnection() errors after LoadBalancer/LBFactory destruction (T147520)]] (duration: 00m 50s) | 
  [production] | 
            
  | 22:01 | 
  <ejegg> | 
  enabled paypal nightly audit parser | 
  [production] | 
            
  | 22:00 | 
  <ori@tin> | 
  Synchronized wmf-config/abusefilter.php: If794eb2a: AbuseFilter: Use new parser from I4aea5f00 on Labs (duration: 00m 49s) | 
  [production] | 
            
  | 21:53 | 
  <ejegg> | 
  updated tools from bde60d5ebbdb89c8ecdf0653e92d29f6a5939ec7 to 6e36fd547b97c426bfb6810d5bc2c9fd4b66efa5 | 
  [production] | 
            
  | 21:53 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/Wikibase-production.php: SWAT: [[gerrit:311206|Add config for units on Wikidata (T117032)]] PART II (duration: 00m 50s) | 
  [production] | 
            
  | 21:51 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/unitConversionConfig.json: SWAT: [[gerrit:311206|Add config for units on Wikidata (T117032)]] PART I (duration: 00m 48s) | 
  [production] | 
            
  | 21:38 | 
  <ejegg> | 
  updated payments-wiki from 27ffd8cb01a5820a0a4e143503b1976e858984bd to 1fbe171bca30d1fb7a4b7d937a740d87f7c9c8e3 | 
  [production] | 
            
  | 21:22 | 
  <mutante> | 
  restarting zuul on gallium | 
  [production] | 
            
  | 21:12 | 
  <ostriches> | 
  lead: enabling & running puppet again, should bring things back up | 
  [production] | 
            
  | 21:02 | 
  <akosiaris> | 
  rebooting lead one more time | 
  [production] | 
            
  | 20:17 | 
  <akosiaris> | 
  rebooting lead once more | 
  [production] | 
            
  | 19:47 | 
  <ostriches> | 
  lead: rebooting, because what have we got to lose | 
  [production] | 
            
  | 19:23 | 
  <twentyafterfour> | 
  disabled phd and puppet on iridium | 
  [production] | 
            
  | 18:31 | 
  <twentyafterfour> | 
  stopped phd on iridium to relieve some load on gerrit | 
  [production] | 
            
  | 18:24 | 
  <ostriches> | 
  lead: restarting apache to force error page to show for now | 
  [production] | 
            
  | 18:21 | 
  <ostriches> | 
  lead: disabled puppet for now, gerrit's sick | 
  [production] | 
            
  | 18:04 | 
  <ostriches> | 
  gerrit: kicking gerrit and apache, something is unhappy... | 
  [production] | 
            
  | 17:11 | 
  <ema> | 
  power-cycling cp2017 | 
  [production] | 
            
  | 16:54 | 
  <akosiaris> | 
  uploaded to apt.wikimedia.org precise-wikimedia/main: php5_5.3.10-1ubuntu3.25+wmf1 | 
  [production] | 
            
  | 16:31 | 
  <ema> | 
  power-cycling cp2022 | 
  [production] | 
            
  | 15:48 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Thu Oct  6 15:48:38 UTC 2016 (duration 7m 12s) | 
  [production] | 
            
  | 15:41 | 
  <dereckson@tin> | 
  scap sync-l10n completed (1.28.0-wmf.21) (duration: 15m 46s) | 
  [production] | 
            
  | 15:41 | 
  <_joe_> | 
  upgrading conftool to 0.3.1 on all mw*, wtp* servers, T147480 T145518 | 
  [production] | 
            
  | 15:25 | 
  <ema> | 
  powercycle cp3045 | 
  [production] | 
            
  | 15:05 | 
  <dereckson@tin> | 
  scap sync-l10n completed (1.28.0-wmf.20) (duration: 15m 59s) | 
  [production] | 
            
  | 15:03 | 
  <ema> | 
  cp3034 hanging during boot, power-cycled | 
  [production] | 
            
  | 14:41 | 
  <jynus> | 
  restarting db1069:3133 mysql instance | 
  [production] | 
            
  | 14:40 | 
  <_joe_> | 
  uploaded conftool 0.3.1 to apt.w.o, T147480 | 
  [production] | 
            
  | 14:33 | 
  <ema> | 
  cache_upload: rolling reboots for kernel upgrades | 
  [production] | 
            
  | 14:26 | 
  <dereckson@tin> | 
  Synchronized php-1.28.0-wmf.21/extensions/WikimediaMessages/i18n/wikimedia: Wikimedia messages for new 'engineer' group for ruwiki (T144599) (duration: 00m 49s) | 
  [production] | 
            
  | 14:25 | 
  <dereckson@tin> | 
  Synchronized php-1.28.0-wmf.20/extensions/WikimediaMessages/i18n/wikimedia: Wikimedia messages for new 'engineer' group for ruwiki (T144599) (duration: 00m 49s) | 
  [production] | 
            
  | 14:23 | 
  <dereckson@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: New 'engineer' group for ruwiki (T144599) (duration: 00m 52s) | 
  [production] | 
            
  | 13:58 | 
  <dereckson@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: Configure Visual Editor namespaces on sv.wikipedia ([[gerrit:309808]] and [[gerrit:314558]], T144688) (duration: 00m 50s) | 
  [production] | 
            
  | 13:54 | 
  <mobrovac> | 
  citoid deploying 4d97774 | 
  [production] | 
            
  | 13:50 | 
  <elukey> | 
  added mw122[67] back to the api appservers live pool | 
  [production] | 
            
  | 13:50 | 
  <ema> | 
  cache_text: rolling reboots for kernel upgrades | 
  [production] | 
            
  | 13:38 | 
  <dereckson@tin> | 
  Synchronized wmf-config/CommonSettings.php: Disable Upload Wizard blacklist issues on Commons (T146417) (duration: 00m 49s) | 
  [production] | 
            
  | 13:23 | 
  <dereckson@tin> | 
  Synchronized php-1.28.0-wmf.21/extensions/EventBus: Send a resource_change event on page_image property change (T145569) (duration: 00m 48s) | 
  [production] | 
            
  | 13:15 | 
  <dereckson@tin> | 
  Synchronized wmf-config/CirrusSearch-common.php: Initialize subphrases autocomplete on wikisources, mw.org and wikitech (T146208, 3/3) (duration: 00m 49s) | 
  [production] | 
            
  | 13:14 | 
  <dereckson@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: Initialize subphrases autocomplete on wikisources, mw.org and wikitech (T146208, 2/3) (duration: 00m 49s) | 
  [production] | 
            
  | 13:11 | 
  <dereckson@tin> | 
  Synchronized tests/cirrusTest.php: Initialize subphrases autocomplete on wikisources, mw.org and wikitech (T146208, 1/3, no-op in prod part) (duration: 00m 50s) | 
  [production] | 
            
  | 12:50 | 
  <ema> | 
  cache_misc: rolling reboots for kernel upgrades | 
  [production] | 
            
  | 12:36 | 
  <elukey> | 
  reimaging mw122[67] to Debian Jessie | 
  [production] | 
            
  | 12:33 | 
  <elukey> | 
  adding mw122[45] back to the live api appservers pool (note: mw1224 was pooled => no before the reimage, but I don't see any blocker in adding it back to serve live traffic) | 
  [production] | 
            
  | 11:57 | 
  <mobrovac> | 
  restbase deploy end of fa4dc79 | 
  [production] | 
            
  | 11:50 | 
  <moritzm> | 
  rebooting video scalers for kernel security update | 
  [production] | 
            
  | 11:36 | 
  <mobrovac> | 
  restbase deploy start of fa4dc79 | 
  [production] | 
            
  | 11:14 | 
  <elukey> | 
  reimaging mw122[34] to Debian Jessie | 
  [production] | 
            
  | 11:12 | 
  <elukey> | 
  added mw122[23] back to the api appservers live pool | 
  [production] | 
            
  | 10:50 | 
  <mobrovac> | 
  change-prop deploying 403eec8 | 
  [production] |