| 
      
        2021-04-08
      
      ยง
     | 
  
    
  | 10:13 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'Depool db1157 for schema change', diff saved to https://phabricator.wikimedia.org/P15253 and previous config saved to /var/cache/conftool/dbconfig/20210408-101303-marostegui.json | 
  [production] | 
            
  | 10:11 | 
  <zpapierski@deploy1002> | 
  Finished deploy [wikimedia/discovery/analytics@ff0137d]: T273847 export queries to relforge dag deployment - start date update (duration: 01m 37s) | 
  [production] | 
            
  | 10:11 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'db1166 (re)pooling @ 100%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15252 and previous config saved to /var/cache/conftool/dbconfig/20210408-101119-root.json | 
  [production] | 
            
  | 10:10 | 
  <mvolz@deploy1002> | 
  helmfile [staging] Ran 'sync' command on namespace 'citoid' for release 'staging' . | 
  [production] | 
            
  | 10:09 | 
  <zpapierski@deploy1002> | 
  Started deploy [wikimedia/discovery/analytics@ff0137d]: T273847 export queries to relforge dag deployment - start date update | 
  [production] | 
            
  | 10:08 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'Add db1180 to dbctl T275633', diff saved to https://phabricator.wikimedia.org/P15251 and previous config saved to /var/cache/conftool/dbconfig/20210408-100829-marostegui.json | 
  [production] | 
            
  | 10:07 | 
  <jgiannelos@deploy1002> | 
  helmfile [codfw] Ran 'sync' command on namespace 'proton' for release 'production' . | 
  [production] | 
            
  | 10:01 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'db1111 (re)pooling @ 75%: Repool db1111', diff saved to https://phabricator.wikimedia.org/P15250 and previous config saved to /var/cache/conftool/dbconfig/20210408-100159-root.json | 
  [production] | 
            
  | 09:58 | 
  <jayme@deploy1002> | 
  helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. | 
  [production] | 
            
  | 09:56 | 
  <jayme@deploy1002> | 
  helmfile [staging-eqiad] START helmfile.d/admin 'apply'. | 
  [production] | 
            
  | 09:56 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'db1166 (re)pooling @ 75%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15249 and previous config saved to /var/cache/conftool/dbconfig/20210408-095615-root.json | 
  [production] | 
            
  | 09:46 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'db1111 (re)pooling @ 50%: Repool db1111', diff saved to https://phabricator.wikimedia.org/P15248 and previous config saved to /var/cache/conftool/dbconfig/20210408-094655-root.json | 
  [production] | 
            
  | 09:44 | 
  <Urbanecm> | 
  [urbanecm@mwmaint1002 ~/uploads]$ mwscript importImages.php --wiki=commonswiki --comment-ext=txt --sleep=3600 --user=Lusccasdeutsch . # T278856 | 
  [production] | 
            
  | 09:42 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'Add db1177 to dbctl T275633', diff saved to https://phabricator.wikimedia.org/P15247 and previous config saved to /var/cache/conftool/dbconfig/20210408-094218-marostegui.json | 
  [production] | 
            
  | 09:42 | 
  <effie> | 
  disable puppet in mw* servers for 677114 | 
  [production] | 
            
  | 09:41 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'db1166 (re)pooling @ 50%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15246 and previous config saved to /var/cache/conftool/dbconfig/20210408-094112-root.json | 
  [production] | 
            
  | 09:36 | 
  <Urbanecm> | 
  Retry server-side upload for T279192 | 
  [production] | 
            
  | 09:31 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'db1111 (re)pooling @ 25%: Repool db1111', diff saved to https://phabricator.wikimedia.org/P15244 and previous config saved to /var/cache/conftool/dbconfig/20210408-093151-root.json | 
  [production] | 
            
  | 09:30 | 
  <moritzm> | 
  installing openssl updates for buster | 
  [production] | 
            
  | 09:29 | 
  <akosiaris@deploy1002> | 
  helmfile [ml-serve-codfw] DONE helmfile.d/admin 'sync'. | 
  [production] | 
            
  | 09:29 | 
  <akosiaris@deploy1002> | 
  helmfile [ml-serve-codfw] START helmfile.d/admin 'sync'. | 
  [production] | 
            
  | 09:27 | 
  <zpapierski@deploy1002> | 
  Finished deploy [wikimedia/discovery/analytics@d098717]: T273847 export queries to relforge dag deployment - sensor name fix (duration: 01m 48s) | 
  [production] | 
            
  | 09:26 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'db1166 (re)pooling @ 25%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15243 and previous config saved to /var/cache/conftool/dbconfig/20210408-092608-root.json | 
  [production] | 
            
  | 09:25 | 
  <zpapierski@deploy1002> | 
  Started deploy [wikimedia/discovery/analytics@d098717]: T273847 export queries to relforge dag deployment - sensor name fix | 
  [production] | 
            
  | 09:24 | 
  <moritzm> | 
  installing libzstd security updates on buster | 
  [production] | 
            
  | 09:20 | 
  <ema> | 
  cp5001: varnish-frontend-restart to test exp policy settings starting from a empty cache T275809 | 
  [production] | 
            
  | 09:14 | 
  <akosiaris@deploy1002> | 
  helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. | 
  [production] | 
            
  | 09:14 | 
  <akosiaris@deploy1002> | 
  helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. | 
  [production] | 
            
  | 09:09 | 
  <moritzm> | 
  installing underscore security updates on stretch | 
  [production] | 
            
  | 08:56 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'Depool db1166 for schema change', diff saved to https://phabricator.wikimedia.org/P15242 and previous config saved to /var/cache/conftool/dbconfig/20210408-085630-marostegui.json | 
  [production] | 
            
  | 08:56 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'db1175 (re)pooling @ 100%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15241 and previous config saved to /var/cache/conftool/dbconfig/20210408-085610-root.json | 
  [production] | 
            
  | 08:44 | 
  <akosiaris@deploy1002> | 
  helmfile [eqiad] DONE helmfile.d/admin 'sync'. | 
  [production] | 
            
  | 08:43 | 
  <akosiaris@deploy1002> | 
  helmfile [eqiad] START helmfile.d/admin 'sync'. | 
  [production] | 
            
  | 08:41 | 
  <akosiaris@deploy1002> | 
  helmfile [codfw] DONE helmfile.d/admin 'sync'. | 
  [production] | 
            
  | 08:41 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'db1175 (re)pooling @ 75%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15240 and previous config saved to /var/cache/conftool/dbconfig/20210408-084107-root.json | 
  [production] | 
            
  | 08:40 | 
  <akosiaris@deploy1002> | 
  helmfile [codfw] START helmfile.d/admin 'sync'. | 
  [production] | 
            
  | 08:39 | 
  <akosiaris@deploy1002> | 
  helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. | 
  [production] | 
            
  | 08:38 | 
  <akosiaris@deploy1002> | 
  helmfile [staging-codfw] START helmfile.d/admin 'sync'. | 
  [production] | 
            
  | 08:38 | 
  <akosiaris@deploy1002> | 
  helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. | 
  [production] | 
            
  | 08:37 | 
  <akosiaris@deploy1002> | 
  helmfile [staging-eqiad] START helmfile.d/admin 'apply'. | 
  [production] | 
            
  | 08:33 | 
  <moritzm> | 
  installing remaining curl security updates for buster | 
  [production] | 
            
  | 08:26 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'db1175 (re)pooling @ 50%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15239 and previous config saved to /var/cache/conftool/dbconfig/20210408-082603-root.json | 
  [production] | 
            
  | 08:24 | 
  <marostegui> | 
  Stop MySQL on all db1117 sections to upgrade kernel | 
  [production] | 
            
  | 08:17 | 
  <moritzm> | 
  imported postgis 3.1.1+dfsg-1~wmf1 to component/postgis for buster-wikimedia T277064 | 
  [production] | 
            
  | 08:15 | 
  <filippo@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be2028.codfw.wmnet | 
  [production] | 
            
  | 08:11 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'db1175 (re)pooling @ 25%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15238 and previous config saved to /var/cache/conftool/dbconfig/20210408-081059-root.json | 
  [production] | 
            
  | 08:03 | 
  <filippo@cumin1001> | 
  START - Cookbook sre.hosts.reboot-single for host ms-be2028.codfw.wmnet | 
  [production] | 
            
  | 07:54 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'es1023 (re)pooling @ 100%: Repool es1023', diff saved to https://phabricator.wikimedia.org/P15237 and previous config saved to /var/cache/conftool/dbconfig/20210408-075457-root.json | 
  [production] | 
            
  | 07:50 | 
  <filippo@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1020.eqiad.wmnet | 
  [production] | 
            
  | 07:49 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'Remove weight from es5 master', diff saved to https://phabricator.wikimedia.org/P15236 and previous config saved to /var/cache/conftool/dbconfig/20210408-074911-marostegui.json | 
  [production] |