| 2016-06-14
      
      § | 
    
  | 15:07 | <thcipriani@tin> | Synchronized dblists/visualeditor-default.dblist: SWAT: [[gerrit:292746|Enable VisualEditor by default on eleven Wikivoyages]] (duration: 01m 49s) | [production] | 
            
  | 13:47 | <hashar> | T136971 Cutting MediaWiki branches 1.28.0-wmf.6 | [production] | 
            
  | 13:40 | <moritzm> | installing apache trusty updates on codfw app servers | [production] | 
            
  | 13:28 | <paravoid> | rebooting install2001, T137647 | [production] | 
            
  | 12:58 | <moritzm> | installing apache trusty updates on canary app servers | [production] | 
            
  | 12:55 | <mobrovac> | change-prop deployed f34fb06c99 | [production] | 
            
  | 12:27 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1024, pool for the first time db1090 with low weight (duration: 00m 38s) | [production] | 
            
  | 11:30 | <mobrovac> | scb disabling puppet for 10 mins or so to keep change-prop down | [production] | 
            
  | 11:15 | <akosiaris> | T134242 rebooting alsafi.wikimedia.org  hassaleh.codfw.wmnet   kraz.wikimedia.org mx2001.wikimedia.org   planet2001.codfw.wmnet   pollux.wikimedia.org   pybal-test2001.codfw.wmnet   pybal-test2002.codfw.wmnet  pybal-test2003.codfw.wmnet for qemu-kvm upgrade | [production] | 
            
  | 11:13 | <akosiaris> | T134242 install qemu-system-common, qemu-system-x86 1:2.5+dfsg-4~bpo8+1 from jessie-backports on ganeti200{1,2,3,4,5,6} | [production] | 
            
  | 11:04 | <_joe_> | pooling all the new codfw appservers that have been installed - mw2215-mw2240 (T135466) | [production] | 
            
  | 10:56 | <_joe_> | pooling the new jessie appservers, mw1263-71 | [production] | 
            
  | 10:52 | <oblivian@palladium> | conftool action : set/weight=30; selector: cluster=appserver,dc=eqiad,name=mw12[67].* | [production] | 
            
  | 09:27 | <godog> | roll-restart swift proxy in codfw and eqiad | [production] | 
            
  | 09:04 | <hashar> | gallium: manually removing cron entry zuul_repack from user zuul. Causes cron spam due to zuul merger no more being on gallium  T137418 | [production] | 
            
  | 08:59 | <jynus> | stopping db1040 for cloning to new s4 hosts | [production] | 
            
  | 08:28 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1040 for cloning (duration: 00m 32s) | [production] | 
            
  | 08:23 | <_joe_> | powercycling mw1154, unresponsive | [production] | 
            
  | 07:19 | <jynus> | powercycling mw1156, could not regain control after OOM | [production] | 
            
  | 07:18 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1024, increase weight of db1082, db1087 and db1092 (duration: 10m 50s) | [production] | 
            
  | 07:05 | <_joe_> | rolling reboot of mw2233-40 | [production] | 
            
  | 06:47 | <_joe_> | rebooting mw2228 | [production] | 
            
  | 06:43 | <_joe_> | rebooting mw2228 | [production] | 
            
  | 06:29 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Pool db1052, db1080, db1083, db1089 (duration: 01m 31s) | [production] | 
            
  | 02:39 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Tue Jun 14 02:39:50 UTC 2016 (duration 5m 59s) | [production] | 
            
  | 02:33 | <mwdeploy@tin> | scap sync-l10n completed (1.28.0-wmf.5) (duration: 12m 14s) | [production] | 
            
  
    | 2016-06-13
      
      § | 
    
  | 23:50 | <dereckson@tin> | Synchronized wmf-config/InitialiseSettings.php: Add ORES to whitelisted beta features (T130211) (duration: 00m 23s) | [production] | 
            
  | 23:42 | <dereckson@tin> | Synchronized php-1.28.0-wmf.5/extensions/ORES/includes/Hooks.php: Update links to beta features (duration: 00m 25s) | [production] | 
            
  | 23:33 | <ejegg> | updated payments from 44102c59ac897c9acab470bf83369d233f9b736f to 2fc573cbb94e833c4144aa9dad79de8ec374bb09 | [production] | 
            
  | 23:29 | <dereckson@tin> | Synchronized wmf-config/InitialiseSettings.php: Update cross-wiki upload configuration ([[Gerrit:293355]]) (duration: 00m 23s) | [production] | 
            
  | 23:10 | <dereckson@tin> | Synchronized portals: (no message) (duration: 00m 24s) | [production] | 
            
  | 23:10 | <dereckson@tin> | Synchronized portals/prod/wikipedia.org/assets: (no message) (duration: 00m 24s) | [production] | 
            
  | 22:51 | <demon@tin> | Synchronized wmf-config/CommonSettings.php: Update extension distributor settings (duration: 00m 24s) | [production] | 
            
  | 22:42 | <yurik> | switched to scap3 and deployed tilerator. Deployed kartotherian. Restarted. | [production] | 
            
  | 22:41 | <dapatrick> | Deployed patches for T129738 to wmf5 | [production] | 
            
  | 22:36 | <awight> | update fundraising CRM revert from e684b7823e751558772a4de4ac23819bc601eb74 to bb9bf136dc0fa82d5d07ebeb33d696e54672b2d6 | [production] | 
            
  | 22:11 | <awight> | Updating fundraising CRM from b7b46740d701942507dca0a98a75f3f87b6b31b1 to e684b7823e751558772a4de4ac23819bc601eb74 | [production] | 
            
  | 19:15 | <aaron@tin> | Synchronized php-1.28.0-wmf.5/resources: ee2da9c2ae6fac93bf65d17b5ea48e5c47c87d47 (duration: 00m 35s) | [production] | 
            
  | 18:20 | <bblack> | upgrading nginx (etc) on deployment-prep caches | [production] | 
            
  | 18:11 | <gehel> | deploying latest GUI on WDQS, | [production] | 
            
  | 17:58 | <urandom> | Upgrade of restbase1007.eqiad.wmnet (https://people.wikimedia.org/~eevans/debian/cassandra_2.2.6-wmf1_all.deb) complete : T137474 | [production] | 
            
  | 17:55 | <urandom> | Restarting restbase1007-c.eqiad.wmnet : T137474 | [production] | 
            
  | 17:52 | <urandom> | Restarting restbase1007-b.eqiad.wmnet : T137474 | [production] | 
            
  | 17:47 | <awight> | Whitelist Special:PaypalExpressGatewayResult | [production] | 
            
  | 17:43 | <godog> | enable proxy_http apache module on graphite1003 / graphite2002 and restart apache | [production] | 
            
  | 17:38 | <urandom> | Restarting restbase1007-a.eqiad.wmnet : T137474 | [production] | 
            
  | 17:37 | <urandom> | Upgrading restbase1007.eqiad.wmnet w/ https://people.wikimedia.org/~eevans/debian/cassandra_2.2.6-wmf1_all.deb : T137474 | [production] | 
            
  | 17:35 | <awight> | update paymentswiki from 63fbe39fbc4d671fd2705ce9e42762b7c49564c2 to 44102c59ac897c9acab470bf83369d233f9b736f | [production] | 
            
  | 16:51 | <_joe_> | powercycling mw1115 | [production] | 
            
  | 16:49 | <thcipriani@tin> | Finished scap: Update l10n cache for ores (duration: 32m 04s) | [production] |