| 
      
        2021-12-01
      
      §
     | 
  
    
  | 08:41 | 
  <oblivian@deploy1002> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 08:32 | 
  <oblivian@deploy1002> | 
  helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 06:20 | 
  <marostegui@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2141.codfw.wmnet with reason: Maintenance T277354 | 
  [production] | 
            
  | 06:20 | 
  <marostegui@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 4:00:00 on db2141.codfw.wmnet with reason: Maintenance T277354 | 
  [production] | 
            
  | 06:20 | 
  <marostegui@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2124.codfw.wmnet with reason: Maintenance T277354 | 
  [production] | 
            
  | 06:20 | 
  <marostegui@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 4:00:00 on db2124.codfw.wmnet with reason: Maintenance T277354 | 
  [production] | 
            
  | 06:19 | 
  <marostegui@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2117.codfw.wmnet with reason: Maintenance T277354 | 
  [production] | 
            
  | 06:19 | 
  <marostegui@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 4:00:00 on db2117.codfw.wmnet with reason: Maintenance T277354 | 
  [production] | 
            
  | 00:35 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 00:35 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 00:32 | 
  <catrope@deploy1002> | 
  Synchronized php-1.38.0-wmf.9/extensions/GrowthExperiments/includes/NewcomerTasks/NewcomerTasksUserOptionsLookup.php: Backport: [[gerrit:742548|Newcomer tasks: Fix filtering of non-existent task types (T296366)]] (duration: 00m 56s) | 
  [production] | 
            
  | 00:10 | 
  <catrope@deploy1002> | 
  Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:742817|Enable A/B test enrollment instrumentation. (T292587)]] (duration: 00m 56s) | 
  [production] | 
            
  | 00:08 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 00:07 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  
    | 
      
        2021-11-30
      
      §
     | 
  
    
  | 23:59 | 
  <dzahn@deploy1002> | 
  helmfile [staging] Ran 'sync' command on namespace 'miscweb' for release 'main' . | 
  [production] | 
            
  | 23:57 | 
  <mutante> | 
  deploy1002 - kube_env miscweb staging ; helmfile -e staging destroy | 
  [production] | 
            
  | 23:56 | 
  <dzahn@deploy1002> | 
  helmfile [staging] Ran 'sync' command on namespace 'miscweb' for release 'main' . | 
  [production] | 
            
  | 23:46 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 23:45 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 23:09 | 
  <mutante> | 
  gerrit - added Majavah to wmf-deployment group for T296777 | 
  [production] | 
            
  | 22:30 | 
  <krinkle@deploy1002> | 
  Finished deploy [integration/docroot@2af7007]: Ia89b6591639e5 (duration: 00m 09s) | 
  [production] | 
            
  | 22:30 | 
  <krinkle@deploy1002> | 
  Started deploy [integration/docroot@2af7007]: Ia89b6591639e5 | 
  [production] | 
            
  | 22:21 | 
  <mutante> | 
  welcome Majavah to MediaWiki deployers (T296777) | 
  [production] | 
            
  | 20:09 | 
  <urbanecm@deploy1002> | 
  Synchronized wmf-config/InitialiseSettings.php: 5443b78f197b782238632966891d721859733a74: uzwiki: Deploy Growth features to newcomers (T294245) (duration: 00m 57s) | 
  [production] | 
            
  | 18:09 | 
  <legoktm> | 
  uploaded php-yaml for component/php72 (T296331) | 
  [production] | 
            
  | 18:08 | 
  <vgutierrez> | 
  restart haproxy on cp3064 - T290005 | 
  [production] | 
            
  | 17:44 | 
  <jynus@cumin1001> | 
  dbctl commit (dc=all): 'Repool db1163 fully', diff saved to https://phabricator.wikimedia.org/P17912 and previous config saved to /var/cache/conftool/dbconfig/20211130-174434-jynus.json | 
  [production] | 
            
  | 17:39 | 
  <jynus@cumin1001> | 
  dbctl commit (dc=all): 'Repool db1163 at 50%', diff saved to https://phabricator.wikimedia.org/P17911 and previous config saved to /var/cache/conftool/dbconfig/20211130-173935-jynus.json | 
  [production] | 
            
  | 17:35 | 
  <jynus@cumin1001> | 
  dbctl commit (dc=all): 'Repool db1163 at 25%', diff saved to https://phabricator.wikimedia.org/P17910 and previous config saved to /var/cache/conftool/dbconfig/20211130-173517-jynus.json | 
  [production] | 
            
  | 17:34 | 
  <moritzm> | 
  installing libvorbis security updates | 
  [production] | 
            
  | 17:15 | 
  <jynus@cumin1001> | 
  dbctl commit (dc=all): 'Repool db1163 at 5%', diff saved to https://phabricator.wikimedia.org/P17908 and previous config saved to /var/cache/conftool/dbconfig/20211130-171550-jynus.json | 
  [production] | 
            
  | 17:00 | 
  <jynus> | 
  move db1139:s1 under db1118 | 
  [production] | 
            
  | 16:57 | 
  <jynus@cumin1001> | 
  dbctl commit (dc=all): 'Depool db1163 fully', diff saved to https://phabricator.wikimedia.org/P17907 and previous config saved to /var/cache/conftool/dbconfig/20211130-165718-jynus.json | 
  [production] | 
            
  | 16:29 | 
  <XioNoX> | 
  Move cr2-codfw lumen transit link to BO cable - T289241 | 
  [production] | 
            
  | 16:26 | 
  <XioNoX> | 
  Move cr2-codfw eqord link to BO cable - T289241 | 
  [production] | 
            
  | 16:23 | 
  <XioNoX> | 
  Move cr2-codfw pfw3 link to BO cable - T289241 | 
  [production] | 
            
  | 16:20 | 
  <Emperor> | 
  reboot ms-be2059 to fix device enumeration order re T295563 | 
  [production] | 
            
  | 16:14 | 
  <jynus@cumin1001> | 
  dbctl commit (dc=all): 'Depool db1163 at 25%', diff saved to https://phabricator.wikimedia.org/P17906 and previous config saved to /var/cache/conftool/dbconfig/20211130-161457-jynus.json | 
  [production] | 
            
  | 16:13 | 
  <XioNoX> | 
  cr2-codfw bounce fpc 1 pic 0 (vrrp backup) - T289241 | 
  [production] | 
            
  | 16:07 | 
  <jynus@cumin1001> | 
  dbctl commit (dc=all): 'Depool db1163 at 50%', diff saved to https://phabricator.wikimedia.org/P17905 and previous config saved to /var/cache/conftool/dbconfig/20211130-160748-jynus.json | 
  [production] | 
            
  | 16:06 | 
  <bblack> | 
  lvs2007 - repooling into service | 
  [production] | 
            
  | 16:01 | 
  <bblack> | 
  lvs2007 - depooling for network maint - do not push LVS config changes please! | 
  [production] | 
            
  | 15:41 | 
  <jbond@cumin1001> | 
  END (FAIL) - Cookbook sre.hosts.decommission (exit_code=99) for hosts puppetboard2001.codfw.wmnet | 
  [production] | 
            
  | 15:41 | 
  <jbond@cumin1001> | 
  START - Cookbook sre.hosts.decommission for hosts puppetboard2001.codfw.wmnet | 
  [production] | 
            
  | 15:38 | 
  <jbond@cumin1001> | 
  END (FAIL) - Cookbook sre.hosts.decommission (exit_code=99) for hosts puppetboard2001.codfw.wmnet | 
  [production] | 
            
  | 15:37 | 
  <jbond@cumin1001> | 
  START - Cookbook sre.hosts.decommission for hosts puppetboard2001.codfw.wmnet | 
  [production] | 
            
  | 15:32 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 15:29 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 15:23 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] | 
            
  | 15:22 | 
  <mwdebug-deploy@deploy1002> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | 
  [production] |