| 2017-11-17
      
      § | 
    
  | 20:35 | <mutante> | netmon1002 - rsync smokeping data back from local backup to show measurements made from eqiad as requested on T180812 | [production] | 
            
  | 20:05 | <mutante> | running puppet on all cache misc to switch smokeping web to eqiad | [production] | 
            
  | 20:02 | <mutante> | T180812 copying smokeping data from 2001 to 1002 - netmon1002: /usr/bin/rsync -avp rsync://netmon2001.wikimedia.org/var-lib-smokeping /var/lib/smokeping/ | switching backend from codfw to eqiad | [production] | 
            
  | 19:37 | <mutante> | T180812 - @netmon1002:# rsync -avp /var/lib/smokeping/ /root/backup/netmon1002/201711717/var/lib/smokeping/@netmon2001:/# rsync -avp /var/lib/smokeping/ /root/backup/netmon2001/201711717/var/lib/smokeping/ | [production] | 
            
  | 19:36 | <mutante> | @netmon1002:/var/lib/smokeping# rsync -avp /var/lib/smokeping/ /root/backup/netmon1002/201711717/var/lib/smokeping/ | [production] | 
            
  | 18:37 | <anomie@tin> | Synchronized wmf-config/InitialiseSettings.php: Setting wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on Beta Cluster, no prod change (duration: 00m 48s) | [production] | 
            
  | 18:36 | <anomie@tin> | Synchronized wmf-config/InitialiseSettings-labs.php: Setting wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on Beta Cluster, no prod change (duration: 00m 49s) | [production] | 
            
  | 17:43 | <demon@tin> | Synchronized php-1.31.0-wmf.8/extensions/Wikidata/extensions/Wikibase/client/WikibaseClient.php: fix client dependencies (duration: 00m 49s) | [production] | 
            
  | 17:41 | <demon@tin> | Synchronized php-1.31.0-wmf.8/extensions/Wikibase/client/WikibaseClient.php: fix client dependencies (duration: 00m 50s) | [production] | 
            
  | 17:21 | <bblack> | enabling new normalization code for all upload@esams (done) | [production] | 
            
  | 17:19 | <bblack> | enabling new normalization code for all upload@eqiad | [production] | 
            
  | 17:14 | <marostegui> | Revert schema change on dbstore1001 - T180714 | [production] | 
            
  | 17:12 | <marostegui> | Move dbstore1001 under db1070 - T180714 | [production] | 
            
  | 17:12 | <bblack> | enabling new normalization code for all upload@codfw | [production] | 
            
  | 16:52 | <bblack> | enabling new normalization code for all upload@ulsfo | [production] | 
            
  | 16:24 | <bblack> | disabling puppet on all cp* (testing encoding patch) | [production] | 
            
  | 16:19 | <moritzm> | uploaded boost 1.55.0+dfsg-3+wmf2+icu57 to apt.wikimedia.org for jessie-wikimedia/component/icu57 (needed for HHVM build linked against ICU 57) | [production] | 
            
  | 16:04 | <dcausse@tin> | Synchronized wmf-config/CirrusSearch-common.php: T180795 [cirrus] disable token count router (duration: 00m 49s) | [production] | 
            
  | 15:43 | <chasemp> | labservices1001:~# mv /var/zones/tools.eqiad.wmflabs /home/rush T180797 | [production] | 
            
  | 15:26 | <urandom> | Starting restbase2006-c w/ -Dcassandra.replace_address=10.192.48.51 (T179422) | [production] | 
            
  | 14:06 | <jynus> | deploy master events to db1070 | [production] | 
            
  | 14:04 | <chasemp> | labstore1003 service nfs-kernel-server restart && service rsync start | [production] | 
            
  | 13:55 | <moritzm> | uploaded boost 1.55.0+dfsg-3+wmf1+icu57 to apt.wikimedia.org for jessie-wikimedia/component/icu57 (needed for HHVM build linked against ICU 57) | [production] | 
            
  | 12:15 | <moritzm> | installing openssl updates on poolcounters | [production] | 
            
  | 12:07 | <moritzm> | installing openssl updates on graphite* hosts | [production] | 
            
  | 11:46 | <moritzm> | installing openssl updates on etcd* hosts | [production] | 
            
  | 11:43 | <moritzm> | installing openssl updates on dbproxy hosts | [production] | 
            
  | 11:33 | <moritzm> | installing openssl updates on puppetmasters | [production] | 
            
  | 11:01 | <akosiaris> | create webperf1001, webperf2001 in ganeti T179036 | [production] | 
            
  | 10:50 | <akosiaris@tin> | Synchronized wmf-config/db-eqiad.php: (no justification provided) (duration: 00m 49s) | [production] | 
            
  | 10:49 | <akosiaris> | sync wmf-config/db-eqiad.php for T180724 | [production] | 
            
  | 10:48 | <akosiaris@tin> | scap aborted: (no justification provided) (duration: 00m 02s) | [production] | 
            
  | 10:48 | <akosiaris@tin> | Started scap: (no justification provided) | [production] | 
            
  | 10:47 | <akosiaris> | pool mw2251 T180724 | [production] | 
            
  | 10:47 | <akosiaris@puppetmaster1001> | conftool action : set/pooled=yes; selector: name=mw2251.codfw.wmnet | [production] | 
            
  | 10:01 | <moritzm> | rebooting darmstadtium (docker registry) for update to 4.9.51 | [production] | 
            
  | 09:59 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Restore original traffic for the s5 eqiad hosts that were out for hours due to schema change revert after s5 master crash (duration: 00m 48s) | [production] | 
            
  | 09:54 | <moritzm> | rebooting labnodepool* for update to 4.9.51 | [production] | 
            
  | 09:38 | <ppchelko@tin> | Finished deploy [cpjobqueue/deploy@2141162]: Bump concurrency even more (duration: 00m 29s) | [production] |