2101-2150 of 10000 results (32ms)
2021-04-08 ยง
11:57 <zpapierski@deploy1002> Started deploy [wikimedia/discovery/analytics@25dad72]: T273847 export queries to relforge dag deployment - elastic index name fix [production]
11:50 <XioNoX> tighten cr3-ulsfo loopback firewall filter - T207799 [production]
11:49 <zpapierski@deploy1002> Finished deploy [wikimedia/discovery/analytics@25dad72]: T273847 export queries to relforge dag deployment - elastic index name fix (duration: 01m 39s) [production]
11:47 <zpapierski@deploy1002> Started deploy [wikimedia/discovery/analytics@25dad72]: T273847 export queries to relforge dag deployment - elastic index name fix [production]
11:46 <marostegui@cumin1001> dbctl commit (dc=all): 'db1157 (re)pooling @ 25%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15261 and previous config saved to /var/cache/conftool/dbconfig/20210408-114625-root.json [production]
11:23 <marostegui@cumin1001> dbctl commit (dc=all): 'db1118 (re)pooling @ 100%: Repool db1118 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P15259 and previous config saved to /var/cache/conftool/dbconfig/20210408-112332-root.json [production]
11:09 <filippo@cumin1001> END (ERROR) - Cookbook sre.hosts.reboot-single (exit_code=97) for host ms-be2028.codfw.wmnet [production]
11:08 <marostegui@cumin1001> dbctl commit (dc=all): 'db1118 (re)pooling @ 75%: Repool db1118 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P15258 and previous config saved to /var/cache/conftool/dbconfig/20210408-110828-root.json [production]
11:07 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: de1670cbd2c59a24f1e29a6d3731e3ac7f39d336: Enable Growth for newcomers on simplewiki, mswiki, tawiki (T278369; T277562; T277550) (duration: 01m 07s) [production]
10:53 <marostegui@cumin1001> dbctl commit (dc=all): 'db1118 (re)pooling @ 50%: Repool db1118 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P15257 and previous config saved to /var/cache/conftool/dbconfig/20210408-105324-root.json [production]
10:47 <effie> disable puppet on parsoid* servers [production]
10:41 <XioNoX> enable sampling on all routers FPCs [production]
10:40 <marostegui> Upgrade db2085's kernel [production]
10:38 <marostegui@cumin1001> dbctl commit (dc=all): 'db1118 (re)pooling @ 25%: Repool db1118 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P15256 and previous config saved to /var/cache/conftool/dbconfig/20210408-103821-root.json [production]
10:37 <mvolz@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'citoid' for release 'production' . [production]
10:32 <XioNoX> enable sampling on cr1-codfw:fpc0 [production]
10:30 <marostegui> Upgrade kernel on db1118 [production]
10:28 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1118 for kernel upgrade', diff saved to https://phabricator.wikimedia.org/P15255 and previous config saved to /var/cache/conftool/dbconfig/20210408-102855-marostegui.json [production]
10:27 <effie> enable puppet on all mw* servers [production]
10:27 <mvolz@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'citoid' for release 'production' . [production]
10:17 <marostegui@cumin1001> dbctl commit (dc=all): 'db1111 (re)pooling @ 100%: Repool db1111', diff saved to https://phabricator.wikimedia.org/P15254 and previous config saved to /var/cache/conftool/dbconfig/20210408-101702-root.json [production]
10:17 <jgiannelos@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'proton' for release 'production' . [production]
10:16 <filippo@cumin1001> START - Cookbook sre.hosts.reboot-single for host ms-be2028.codfw.wmnet [production]
10:13 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1157 for schema change', diff saved to https://phabricator.wikimedia.org/P15253 and previous config saved to /var/cache/conftool/dbconfig/20210408-101303-marostegui.json [production]
10:11 <zpapierski@deploy1002> Finished deploy [wikimedia/discovery/analytics@ff0137d]: T273847 export queries to relforge dag deployment - start date update (duration: 01m 37s) [production]
10:11 <marostegui@cumin1001> dbctl commit (dc=all): 'db1166 (re)pooling @ 100%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15252 and previous config saved to /var/cache/conftool/dbconfig/20210408-101119-root.json [production]
10:10 <mvolz@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'citoid' for release 'staging' . [production]
10:09 <zpapierski@deploy1002> Started deploy [wikimedia/discovery/analytics@ff0137d]: T273847 export queries to relforge dag deployment - start date update [production]
10:08 <marostegui@cumin1001> dbctl commit (dc=all): 'Add db1180 to dbctl T275633', diff saved to https://phabricator.wikimedia.org/P15251 and previous config saved to /var/cache/conftool/dbconfig/20210408-100829-marostegui.json [production]
10:07 <jgiannelos@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'proton' for release 'production' . [production]
10:01 <marostegui@cumin1001> dbctl commit (dc=all): 'db1111 (re)pooling @ 75%: Repool db1111', diff saved to https://phabricator.wikimedia.org/P15250 and previous config saved to /var/cache/conftool/dbconfig/20210408-100159-root.json [production]
09:58 <jayme@deploy1002> helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. [production]
09:56 <jayme@deploy1002> helmfile [staging-eqiad] START helmfile.d/admin 'apply'. [production]
09:56 <marostegui@cumin1001> dbctl commit (dc=all): 'db1166 (re)pooling @ 75%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15249 and previous config saved to /var/cache/conftool/dbconfig/20210408-095615-root.json [production]
09:46 <marostegui@cumin1001> dbctl commit (dc=all): 'db1111 (re)pooling @ 50%: Repool db1111', diff saved to https://phabricator.wikimedia.org/P15248 and previous config saved to /var/cache/conftool/dbconfig/20210408-094655-root.json [production]
09:44 <Urbanecm> [urbanecm@mwmaint1002 ~/uploads]$ mwscript importImages.php --wiki=commonswiki --comment-ext=txt --sleep=3600 --user=Lusccasdeutsch . # T278856 [production]
09:42 <marostegui@cumin1001> dbctl commit (dc=all): 'Add db1177 to dbctl T275633', diff saved to https://phabricator.wikimedia.org/P15247 and previous config saved to /var/cache/conftool/dbconfig/20210408-094218-marostegui.json [production]
09:42 <effie> disable puppet in mw* servers for 677114 [production]
09:41 <marostegui@cumin1001> dbctl commit (dc=all): 'db1166 (re)pooling @ 50%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15246 and previous config saved to /var/cache/conftool/dbconfig/20210408-094112-root.json [production]
09:36 <Urbanecm> Retry server-side upload for T279192 [production]
09:31 <marostegui@cumin1001> dbctl commit (dc=all): 'db1111 (re)pooling @ 25%: Repool db1111', diff saved to https://phabricator.wikimedia.org/P15244 and previous config saved to /var/cache/conftool/dbconfig/20210408-093151-root.json [production]
09:30 <moritzm> installing openssl updates for buster [production]
09:29 <akosiaris@deploy1002> helmfile [ml-serve-codfw] DONE helmfile.d/admin 'sync'. [production]
09:29 <akosiaris@deploy1002> helmfile [ml-serve-codfw] START helmfile.d/admin 'sync'. [production]
09:27 <zpapierski@deploy1002> Finished deploy [wikimedia/discovery/analytics@d098717]: T273847 export queries to relforge dag deployment - sensor name fix (duration: 01m 48s) [production]
09:26 <marostegui@cumin1001> dbctl commit (dc=all): 'db1166 (re)pooling @ 25%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15243 and previous config saved to /var/cache/conftool/dbconfig/20210408-092608-root.json [production]
09:25 <zpapierski@deploy1002> Started deploy [wikimedia/discovery/analytics@d098717]: T273847 export queries to relforge dag deployment - sensor name fix [production]
09:24 <moritzm> installing libzstd security updates on buster [production]
09:20 <ema> cp5001: varnish-frontend-restart to test exp policy settings starting from a empty cache T275809 [production]
09:14 <akosiaris@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]