| 
      
        2017-06-30
      
      §
     | 
  
    
  | 06:38 | 
  <marostegui> | 
  Deploy alter table on s6 all eqiad hosts (primary master not included) - T168661 | 
  [production] | 
            
  | 06:12 | 
  <marostegui> | 
  Deploy alter table on db1018 on s2 - T168661 | 
  [production] | 
            
  | 06:12 | 
  <marostegui> | 
  Deploy alter table on db1090 on s2 - T168661 | 
  [production] | 
            
  | 06:11 | 
  <marostegui> | 
  Deploy alter table on db1076 on s2 - T168661 | 
  [production] | 
            
  | 06:10 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1060 - T168661 (duration: 00m 42s) | 
  [production] | 
            
  | 06:10 | 
  <marostegui> | 
  Deploy alter table on db1074 on s2 - T168661 | 
  [production] | 
            
  | 06:07 | 
  <marostegui> | 
  Deploy alter table on db1060 on s2 - T168661 | 
  [production] | 
            
  | 06:06 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1060 - T168661 (duration: 00m 42s) | 
  [production] | 
            
  | 05:59 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1036 - T168661 (duration: 00m 43s) | 
  [production] | 
            
  | 05:56 | 
  <marostegui> | 
  Deploy alter table on db1047 on s2 - T168661 | 
  [production] | 
            
  | 05:56 | 
  <marostegui> | 
  Deploy alter table on db1036 on s2 - T168661 | 
  [production] | 
            
  | 05:55 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1036 - T168661 (duration: 00m 42s) | 
  [production] | 
            
  | 05:47 | 
  <marostegui> | 
  Deploy alter table on db1021 on s2 - T168661 | 
  [production] | 
            
  | 05:44 | 
  <marostegui> | 
  Deploy alter table on dbstore1002 on s2 - T168661 | 
  [production] | 
            
  | 05:43 | 
  <marostegui> | 
  Deploy alter table on dbstore1001 on s2 - T168661 | 
  [production] | 
            
  | 05:37 | 
  <marostegui> | 
  Deploy alter table on db1069 (and let it replicate) on s2 - T168661 | 
  [production] | 
            
  | 02:29 | 
  <chasemp> | 
  labstore1005 start drbd | 
  [production] | 
            
  | 02:14 | 
  <chasemp> | 
  reboot labstore1005 (5m ago) | 
  [production] | 
            
  | 01:25 | 
  <chasemp> | 
  reboot labstoer1005 | 
  [production] | 
            
  | 01:23 | 
  <chasemp> | 
  fail nfs from labstore1005 to labstore1004 (I failed to log a previous failover to 1004 and back) | 
  [production] | 
            
  
    | 
      
        2017-06-29
      
      §
     | 
  
    
  | 23:32 | 
  <RoanKattouw> | 
  Sorry I meant T169163 | 
  [production] | 
            
  | 23:31 | 
  <catrope@tin> | 
  Synchronized php-1.30.0-wmf.7/resources/src/mediawiki.rcfilters/: RCFilters fixes (T169169, T169107, T169042) (duration: 00m 42s) | 
  [production] | 
            
  | 23:28 | 
  <catrope@tin> | 
  Synchronized php-1.30.0-wmf.7/extensions/WikimediaEvents/: Add event logging for explode-similar on SRP (T149809) (duration: 00m 42s) | 
  [production] | 
            
  | 23:26 | 
  <catrope@tin> | 
  Synchronized php-1.30.0-wmf.7/extensions/CirrusSearch/: "Explore similar" widget for CirrusSearch (T149809) (duration: 00m 54s) | 
  [production] | 
            
  | 23:19 | 
  <catrope@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: Stop reader surveys (T131949) (duration: 00m 43s) | 
  [production] | 
            
  | 22:16 | 
  <chasemp> | 
  set cfq scheduler on labstore1005 | 
  [production] | 
            
  | 21:51 | 
  <mutante> | 
  APT - uploading python-django-south from jessie to wikimedia-stretch for librenms on stretch (T159756) | 
  [production] | 
            
  | 21:40 | 
  <chasemp> | 
  reboot labstore1004 with grub set to gnulinux-advanced-1773f282-5a1b-441e-865c-8b70a0ebc925>gnulinux-4.4.0-3-amd64-advanced-1773f282-5a1b-441e-865c-8b70a0ebc925 | 
  [production] | 
            
  | 21:10 | 
  <mobrovac@tin> | 
  Finished deploy [restbase/deploy@bcb83f4]: (no justification provided) (duration: 04m 21s) | 
  [production] | 
            
  | 21:06 | 
  <mobrovac@tin> | 
  Started deploy [restbase/deploy@bcb83f4]: (no justification provided) | 
  [production] | 
            
  | 21:06 | 
  <mobrovac@tin> | 
  Finished deploy [restbase/deploy@bcb83f4]: (no justification provided) (duration: 01m 02s) | 
  [production] | 
            
  | 21:05 | 
  <mobrovac@tin> | 
  Started deploy [restbase/deploy@bcb83f4]: (no justification provided) | 
  [production] | 
            
  | 21:04 | 
  <mobrovac@tin> | 
  Finished deploy [restbase/deploy@bcb83f4]: Fix special char handling in PDF back-end requests - T169223 (duration: 03m 14s) | 
  [production] | 
            
  | 21:00 | 
  <mobrovac@tin> | 
  Started deploy [restbase/deploy@bcb83f4]: Fix special char handling in PDF back-end requests - T169223 | 
  [production] | 
            
  | 20:46 | 
  <mutante> | 
  APT - reprepro copy stretch-wikimedia jessie-wikimedia prometheus-snmp-exporter (to make it available on stretch for netmon1002) (T159756) | 
  [production] | 
            
  | 20:39 | 
  <ppchelko@tin> | 
  Finished deploy [changeprop/deploy@350076c]: Config: Enable red links processing. T133221 (duration: 01m 01s) | 
  [production] | 
            
  | 20:38 | 
  <ppchelko@tin> | 
  Started deploy [changeprop/deploy@350076c]: Config: Enable red links processing. T133221 | 
  [production] | 
            
  | 19:23 | 
  <demon@tin> | 
  Synchronized scap/plugins/clean.py: Because I need to learn basic python syntax before trying stuff (duration: 00m 42s) | 
  [production] | 
            
  | 19:21 | 
  <twentyafterfour@tin> | 
  rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.30.0-wmf.7 refs T167536 | 
  [production] | 
            
  | 19:16 | 
  <twentyafterfour> | 
  deploying wmf/1.30.0-wmf.7 to all wikis refs T167536 | 
  [production] | 
            
  | 19:06 | 
  <demon@tin> | 
  Pruned MediaWiki: 1.30.0-wmf.5 [keeping static files] (duration: 01m 16s) | 
  [production] | 
            
  | 18:30 | 
  <chasemp> | 
  restart nfs on labstore1004 (primary) | 
  [production] | 
            
  | 18:10 | 
  <demon@tin> | 
  Synchronized php-1.30.0-wmf.7/extensions/TextExtracts/extension.json: T107206 (duration: 00m 47s) | 
  [production] | 
            
  | 17:28 | 
  <arlolra> | 
  Updated Parsoid to b4187f18 (T168900, T168675, T168404, T153203) | 
  [production] | 
            
  | 17:21 | 
  <arlolra@tin> | 
  Finished deploy [parsoid/deploy@717df08]: Updating Parsoid to b4187f18 (duration: 09m 41s) | 
  [production] | 
            
  | 17:12 | 
  <arlolra@tin> | 
  Started deploy [parsoid/deploy@717df08]: Updating Parsoid to b4187f18 | 
  [production] | 
            
  | 17:08 | 
  <mobrovac> | 
  scb2005 repooling back the services - T167763 | 
  [production] | 
            
  | 16:21 | 
  <godog> | 
  temporarily stop ircecho, puppet spam | 
  [production] | 
            
  | 16:05 | 
  <akosiaris@tin> | 
  Synchronized wmf-config/ProductionServices.php: (no justification provided) (duration: 00m 46s) | 
  [production] | 
            
  | 15:40 | 
  <akosiaris> | 
  disable puppet on all of eqiad/esams, problems with ganeti and puppetdb | 
  [production] |