| 2019-10-23
      
      § | 
    
  | 08:28 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Repool db2091:3312 after table compression', diff saved to https://phabricator.wikimedia.org/P9452 and previous config saved to /var/cache/conftool/dbconfig/20191023-082826-marostegui.json | [production] | 
            
  | 08:23 | <godog> | roll restart logstash in codfw/eqiad to pick up new kafka partitions | [production] | 
            
  | 08:22 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Change weights to x100 on s8 eqiad - T231018', diff saved to https://phabricator.wikimedia.org/P9451 and previous config saved to /var/cache/conftool/dbconfig/20191023-082246-marostegui.json | [production] | 
            
  | 08:11 | <godog> | kafka-logging eqiad set 12 partitions for ^mwlog- ^logback- and eqiad.client.error topics | [production] | 
            
  | 08:09 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Change weights to x100 on s8 codfw - T231018', diff saved to https://phabricator.wikimedia.org/P9450 and previous config saved to /var/cache/conftool/dbconfig/20191023-080857-marostegui.json | [production] | 
            
  | 07:55 | <godog> | kafka-logging delete unused topic syslog-notice | [production] | 
            
  | 07:51 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Change weights to x100 on s7 eqiad - T231018', diff saved to https://phabricator.wikimedia.org/P9449 and previous config saved to /var/cache/conftool/dbconfig/20191023-075106-marostegui.json | [production] | 
            
  | 07:48 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Change weights to x100 on s7 codfw - T231018', diff saved to https://phabricator.wikimedia.org/P9448 and previous config saved to /var/cache/conftool/dbconfig/20191023-074828-marostegui.json | [production] | 
            
  | 07:46 | <XioNoX> | powering down cr2-esams for relocation (for real this time) | [production] | 
            
  | 07:38 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Change weights to x100 on s6 eqiad - T231018', diff saved to https://phabricator.wikimedia.org/P9447 and previous config saved to /var/cache/conftool/dbconfig/20191023-073831-marostegui.json | [production] | 
            
  | 07:35 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Change weights to x100 on s6 codfw - T231018', diff saved to https://phabricator.wikimedia.org/P9446 and previous config saved to /var/cache/conftool/dbconfig/20191023-073556-marostegui.json | [production] | 
            
  | 07:30 | <XioNoX> | powering down cr2-esams for relocation | [production] | 
            
  | 07:28 | <hashar> | logstash: refreshing index fields for logstash-* indices (via https://logstash.wikimedia.org/app/kibana#/management/kibana/indices/logstash-* ) # T234564 | [production] | 
            
  | 07:05 | <XioNoX> | redirect ns2 to eqiad - T235805 | [production] | 
            
  | 07:04 | <marostegui> | Enable slow query log 1/10 on db1089 (enwiki) T223151 | [production] | 
            
  | 07:02 | <ayounsi@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 07:02 | <ayounsi@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 06:59 | <XioNoX> | depool esams - T235805 | [production] | 
            
  | 06:57 | <effie> | Depooling mw1317 | [production] | 
            
  | 06:54 | <marostegui@cumin1001> | END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) | [production] | 
            
  | 06:54 | <marostegui@cumin1001> | START - Cookbook sre.hosts.decommission | [production] | 
            
  | 06:46 | <marostegui@cumin1001> | END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) | [production] | 
            
  | 06:46 | <marostegui@cumin1001> | START - Cookbook sre.hosts.decommission | [production] | 
            
  | 06:38 | <marostegui> | Compress tables on db1097:3315 T235599 | [production] | 
            
  | 06:38 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Depool db1097:3315 for compression T235599', diff saved to https://phabricator.wikimedia.org/P9445 and previous config saved to /var/cache/conftool/dbconfig/20191023-063800-marostegui.json | [production] | 
            
  | 05:29 | <ema@puppetmaster1001> | conftool action : set/pooled=false; selector: dnsdisc=kibana,name=codfw | [production] | 
            
  | 05:29 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Fully repool db1096:3315 after maintenance maintenance', diff saved to https://phabricator.wikimedia.org/P9444 and previous config saved to /var/cache/conftool/dbconfig/20191023-052940-marostegui.json | [production] | 
            
  | 05:08 | <marostegui@cumin1001> | dbctl commit (dc=all): 'More traffic to db1096:3315 after maintenance maintenance', diff saved to https://phabricator.wikimedia.org/P9443 and previous config saved to /var/cache/conftool/dbconfig/20191023-050812-marostegui.json | [production] | 
            
  | 04:57 | <marostegui@cumin1001> | dbctl commit (dc=all): 'More traffic to db1096:3315 after maintenance maintenance', diff saved to https://phabricator.wikimedia.org/P9442 and previous config saved to /var/cache/conftool/dbconfig/20191023-045722-marostegui.json | [production] | 
            
  | 04:49 | <vgutierrez> | repool cp5007 - T234887 | [production] | 
            
  | 04:48 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Slowly repool db1096:3315 after maintenance maintenance', diff saved to https://phabricator.wikimedia.org/P9441 and previous config saved to /var/cache/conftool/dbconfig/20191023-044833-marostegui.json | [production] | 
            
  | 04:36 | <MaxSem> | Fixed a page title via namespaceDupes.php on pswiki | [production] | 
            
  | 03:51 | <vgutierrez> | depool cp5007 - T234887 | [production] | 
            
  
    | 2019-10-22
      
      § | 
    
  | 23:57 | <maxsem@deploy1001> | Synchronized php-1.35.0-wmf.3/includes/block/DatabaseBlock.php: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/545373/ (duration: 00m 59s) | [production] | 
            
  | 23:53 | <maxsem@deploy1001> | Synchronized wmf-config/wikitech.php: https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/543943/ (duration: 01m 01s) | [production] | 
            
  | 23:43 | <maxsem@deploy1001> | Synchronized dblists/: https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/543664/ (duration: 00m 59s) | [production] | 
            
  | 23:41 | <maxsem@deploy1001> | Synchronized wmf-config: https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/543664/ (duration: 01m 01s) | [production] | 
            
  | 23:38 | <maxsem@deploy1001> | Synchronized dblists/labtestwiki.dblist: https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/543664/ (duration: 01m 02s) | [production] | 
            
  | 23:32 | <mutante> | LDAP - added keepit-ssh to wmf group (T236209) | [production] | 
            
  | 22:23 | <ejegg> | updated Fundraising CiviCRM from ff69d64ad4 to 47e0800001 | [production] | 
            
  | 21:57 | <thcipriani> | stopping gerrit to run ref-update script T236114 | [production] | 
            
  | 21:57 | <thcipriani> | stopping gerrit to run ref-update script | [production] | 
            
  | 21:45 | <mutante> | LDAP - added lexnasser to nda group (T235688) | [production] | 
            
  | 21:07 | <eileen> | process-control config revision is 95ee1bafb3 dedupe job re-enabled | [production] | 
            
  | 20:09 | <mutante> | gerrit1001 - mkdir /srv/gerrit/cobalt/git - rsyncing /srv/gerrit/git from cobalt to /srv/gerrit/cobalt/git/ on gerrit1001 (T236114) | [production] | 
            
  | 19:42 | <hashar> | gerrit1001: apt install colordiff # T236114 | [production] | 
            
  | 19:27 | <brennen@deploy1001> | rebuilt and synchronized wikiversions files: group0 to 1.35.0-wmf.3 | [production] | 
            
  | 19:03 | <brennen> | proceeding with train for 1.35.0-wmf.3 | [production] | 
            
  | 18:09 | <mutante> | DNS - added new Wikipedia language "mnw" (Mon) T235739 - a language spoken in Myanmar | [production] | 
            
  | 17:59 | <sbassett> | Uploaded and applied (but did not deploy per releng) security fix for T234450 to wmf.3 | [production] |