| 2018-01-17
      
      ยง | 
    
  | 20:35 | <pnorman@tin> | Finished deploy [kartotherian/deploy@ecdda41]: (no justification provided) (duration: 05m 44s) | [production] | 
            
  | 20:30 | <pnorman@tin> | Started deploy [kartotherian/deploy@ecdda41]: (no justification provided) | [production] | 
            
  | 20:05 | <andrewbogott> | rebooted labservices1002, labcontrol1002, labnet1002 | [production] | 
            
  | 19:56 | <andrewbogott> | rebooting labpuppetmaster1001 | [production] | 
            
  | 19:46 | <andrewbogott> | rebooting labpuppetmaster1002 | [production] | 
            
  | 19:45 | <papaul> | Powering down mw2140 for main board replacement | [production] | 
            
  | 18:20 | <niharika29@tin> | Synchronized php-1.31.0-wmf.17/includes/EditPage.php: Update Save/Publish button flag from 'constructive' to 'progressive' https://gerrit.wikimedia.org/r/#/c/404733/ (duration: 01m 14s) | [production] | 
            
  | 18:09 | <moritzm> | uploading HHVM 3.18.5+wmf4 for stretch-wikimedia to apt.wikimedia.org (3.18.7 with the patch https://github.com/facebook/hhvm/commit/bd7b2bcfe70b053a3a001480653012f68599250f backed out) | [production] | 
            
  | 18:08 | <ejegg> | turned off main silverpop recipient data fetch job | [production] | 
            
  | 17:55 | <mutante> | gerrit login page design changed (https://gerrit.wikimedia.org/r/402665) in case you were worried it was a fake page trying to steal your login, heh | [production] | 
            
  | 17:44 | <moritzm> | resetting RAC on labsdb1004 (serial console inaccessible) | [production] | 
            
  | 17:17 | <chasemp> | reboot labstore2003 | [production] | 
            
  | 17:12 | <madhuvishy> | Rebooting labstore2004 | [production] | 
            
  | 17:08 | <godog> | bootstrap cassandra-a on restbase1013 | [production] | 
            
  | 17:06 | <ema> | upgrade pybal on primary LVSs to 1.14.3 T184715, T184721 | [production] | 
            
  | 16:52 | <ema> | upgrade secondary LVSs to pybal 1.13.4 T184715, T184721 | [production] | 
            
  | 16:33 | <XioNoX> | routing ns2 to radon | [production] | 
            
  | 16:26 | <ema> | reboot baham (codfw authdns) for kernel upgrade | [production] | 
            
  | 16:24 | <XioNoX> | routing ns1 to eqiad | [production] | 
            
  | 16:17 | <chasemp> | labmon1001:~# service grafana-server | [production] | 
            
  | 16:17 | <ema> | reboot radon (eqiad authdns) for kernel upgrade | [production] | 
            
  | 16:13 | <jgleeson> | updated civicrm from 354f32fe8a to c70f01cd83 | [production] | 
            
  | 16:12 | <chasemp> | labmon1001:~# /sbin/reboot | [production] | 
            
  | 16:09 | <XioNoX> | routing ns0 to codfw (baham) | [production] | 
            
  | 16:07 | <moritzm> | upgrading HHVM in codfw to 3.18.7 (wmf4) | [production] | 
            
  | 16:06 | <moritzm> | upgrading nginx on mwdebug servers to 1.13.6-2+wmf1~jessie1 | [production] | 
            
  | 16:05 | <jgleeson> | turned off donations queue consumer process-control job | [production] | 
            
  | 16:00 | <ema> | pybal 1.14.3 uploaded to apt.w.o | [production] | 
            
  | 15:51 | <chasemp> | labstore1002:~# /sbin/reboot | [production] | 
            
  | 15:41 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1065 after fixing data drifts - T162807 (duration: 01m 12s) | [production] | 
            
  | 15:41 | <_joe_> | dropping ruwiki htmlCacheUpdate records stuck int he old jobqueue | [production] | 
            
  | 15:36 | <moritzm> | upgrading nginx on mw servers in codfw to 1.13.6-2+wmf1~jessie1 | [production] | 
            
  | 15:32 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Fully repool db1104 (duration: 01m 12s) | [production] | 
            
  | 14:57 | <moritzm> | resetting RAC on labsdb1007 (serial console inaccessible) | [production] | 
            
  | 14:53 | <moritzm> | resetting RAC on labsdb1006 (serial console inaccessible) | [production] | 
            
  | 14:38 | <chasemp> | labstore1001:~# /sbin/reboot | [production] | 
            
  | 14:27 | <zeljkof> | EU SWAT finished | [production] | 
            
  | 14:23 | <zfilipin@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:404327|Create "eliminator" user group on ur.wikipedia (T184607)]] (duration: 01m 12s) | [production] | 
            
  | 14:14 | <moritzm> | repooling chromium | [production] | 
            
  | 14:14 | <zfilipin@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:404624|Add Draft Namespace in enwikiversity (T184957)]] (duration: 01m 12s) | [production] | 
            
  | 14:07 | <moritzm> | rebooting chromium for kernel security update | [production] | 
            
  | 14:04 | <gehel> | restart of elasticsearch / cirrus eqiad completed (cluster still recovering) | [production] | 
            
  | 14:03 | <moritzm> | depooling chromium | [production] | 
            
  | 13:51 | <chasemp> | reboot labstore2003 | [production] | 
            
  | 13:46 | <akosiaris> | reboot sca2003 webperf2001 planet2001 poolcounter2002 mx2001 kubetcd200{1,2,3} install2002 dbmonitor2001 alsafi acrux hassaleh diadem nihal pybal-test200{1,2,3} releases2001 tureis for PCID, INVPCID | [production] | 
            
  | 13:45 | <chasemp> | labstore2002:~# sudo update-grub && /sbin/reboot | [production] | 
            
  | 13:40 | <chasemp> | labstore2001:~# /sbin/reboot | [production] | 
            
  | 13:39 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Slowly repool db1104 (duration: 01m 13s) | [production] | 
            
  | 13:31 | <akosiaris> | reboot acrab for PCID,INVPCID enabling | [production] | 
            
  | 13:22 | <marostegui> | Deploy schema change on db1099:3318 - https://phabricator.wikimedia.org/T174569 | [production] |