| 2016-12-20
      
      § | 
    
  | 23:46 | <mutante> | mw1168 - re-signing new puppet cert after install, initial puppet run | [production] | 
            
  | 23:18 | <mutante> | rebooting mw1168 into PXE for reinstall | [production] | 
            
  | 23:09 | <mutante> | install1001/2001 - upgrade python-requests python-urllib3 arcconf | [production] | 
            
  | 21:48 | <mutante> | depooled mw1168 videoscaler for reinstall (part of T153488) | [production] | 
            
  | 21:47 | <dzahn@puppetmaster1001> | conftool action : set/pooled=no; selector: name=mw1168.eqiad.wmnet | [production] | 
            
  | 21:43 | <demon@tin> | Synchronized scap/plugins: no-op scap stuff (duration: 00m 39s) | [production] | 
            
  | 21:42 | <demon@tin> | Synchronized multiversion/submodules.json: no-op scap stuff (duration: 00m 39s) | [production] | 
            
  | 21:08 | <eileen1> | disable Name match only job as we are caught up now | [production] | 
            
  | 21:05 | <eileen1> | disable Major gifts dedupe jobs. The main job has caught up to the end of the queue now so we don't need to target these separately | [production] | 
            
  | 20:55 | <mobrovac@tin> | Finished deploy [changeprop/deploy@77786a5]: Config change: respect the pages blacklisted by RESTBase (duration: 00m 50s) | [production] | 
            
  | 20:54 | <mobrovac@tin> | Starting deploy [changeprop/deploy@77786a5]: Config change: respect the pages blacklisted by RESTBase | [production] | 
            
  | 20:25 | <demon@tin> | Synchronized wmf-config/InitialiseSettings.php: Remove dumb comment (duration: 00m 39s) | [production] | 
            
  | 20:21 | <demon@tin> | Synchronized scap/plugins: no-op scap stuff (duration: 00m 39s) | [production] | 
            
  | 20:21 | <demon@tin> | Synchronized tox.ini: no-op scap stuff (duration: 00m 39s) | [production] | 
            
  | 20:20 | <demon@tin> | Synchronized test-requirements.txt: no-op scap stuff (duration: 00m 39s) | [production] | 
            
  | 20:19 | <demon@tin> | Synchronized setup.py: no-op scap stuff (duration: 00m 39s) | [production] | 
            
  | 20:19 | <demon@tin> | Synchronized requirements.txt: no-op scap stuff (duration: 00m 39s) | [production] | 
            
  | 20:13 | <demon@tin> | Synchronized wmf-config/InitialiseSettings.php: No-op comment-only (duration: 00m 52s) | [production] | 
            
  | 17:50 | <volans> | rcs1002.eqiad.wmnet: python-gevent downgraded from 1.1b6-1~trusty1 to 1.0-1ubuntu1 | [production] | 
            
  | 17:41 | <ema> | rcs1001.eqiad.wmnet: python-gevent downgraded from 1.1b6-1~trusty1 to 1.0-1ubuntu1 | [production] | 
            
  | 17:24 | <reedy@tin> | Synchronized wmf-config/throttle.php: T153741 (duration: 00m 40s) | [production] | 
            
  | 16:31 | <oblivian@puppetmaster1001> | conftool action : set/pooled=yes; selector: cluster=restbase,dc=eqiad,name=restbase101[6-8].* | [production] | 
            
  | 16:30 | <bblack> | upgrading tons of outdated OS packages on rcs100x | [production] | 
            
  | 16:29 | <oblivian@puppetmaster1001> | conftool action : set/pooled=yes; selector: cluster=restbase,dc=codfw,name=restbase201.* | [production] | 
            
  | 16:26 | <marostegui> | Running optimize table on db1045 for the revision tables as we urgently need some space back on that host - T153739 | [production] | 
            
  | 16:19 | <mutante> | notebook1001/1002 - upgraded php packages | [production] | 
            
  | 16:17 | <jynus> | drop repl_records_partition events from db1069 | [production] | 
            
  | 16:09 | <mobrovac> | change-prop restarting everywhere | [production] | 
            
  | 15:59 | <mobrovac> | restarting change-prop on scb1001 | [production] | 
            
  | 15:48 | <mobrovac@tin> | Finished deploy [parsoid/deploy@e8f0865]: Deploy of mwApiServer across the wtp fleet (duration: 05m 59s) | [production] | 
            
  | 15:42 | <mobrovac@tin> | Starting deploy [parsoid/deploy@e8f0865]: Deploy of mwApiServer across the wtp fleet | [production] | 
            
  | 15:32 | <mobrovac@tin> | Finished deploy [parsoid/deploy@e8f0865]: Canary deploy of mwApiServer to wtp[12]001, take two (duration: 00m 37s) | [production] | 
            
  | 15:31 | <mobrovac@tin> | Starting deploy [parsoid/deploy@e8f0865]: Canary deploy of mwApiServer to wtp[12]001, take two | [production] | 
            
  | 14:33 | <mobrovac@tin> | Finished deploy [parsoid/deploy@1d75b14]: Canary deploy of mwApiServer to wtp[12]001 (duration: 00m 32s) | [production] | 
            
  | 14:33 | <mobrovac@tin> | Starting deploy [parsoid/deploy@1d75b14]: Canary deploy of mwApiServer to wtp[12]001 | [production] | 
            
  | 12:04 | <jynus> | running redact_sanitarium.sh on db1069 | [production] | 
            
  | 08:27 | <elukey> | renamed some log files ($something.1.gz to $something.1a.gz) on cp1008 and rutherium to unblock logrotation and reduce cronspam - T132324 | [production] | 
            
  | 02:29 | <mutante> | fermium (lists) - upgrade apache2 | uranium (ganglia-web) - upgrade apache2, openssh, openssl | [production] | 
            
  | 02:26 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Tue Dec 20 02:26:08 UTC 2016 (duration 4m 48s) | [production] | 
            
  | 02:23 | <mutante> | osmium - upgrade imagemagick, php5 packages | hafnium - upgrade php5 | [production] | 
            
  | 02:21 | <l10nupdate@tin> | scap sync-l10n completed (1.29.0-wmf.6) (duration: 06m 41s) | [production] | 
            
  | 02:17 | <mutante> | contint1002/2001, einsteinium/tegmen - upgrade php5 | bromine - upgrade php5, libs | [production] | 
            
  | 02:11 | <mutante> | auth (yubiauth), phab2001 - upgrade php5 | bohrium (piwik) - upgrade ssh,ssl,php5 | [production] | 
            
  | 02:02 | <mutante> | copper - upgraded php5* | krypton - installed all pending upgrades | [production] | 
            
  | 01:45 | <mutante> | terbium - upgrade imagemagick | wasat - upgrade php5, imagemagick | [production] | 
            
  | 01:41 | <mutante> | tin - upgrading php, imagemagick,.. | [production] | 
            
  | 01:35 | <mutante> | mira - upgrading php5 packages | [production] |