1301-1350 of 10000 results (75ms)
2023-01-23 ยง
16:58 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db2114.codfw.wmnet with reason: Maintenance [production]
16:56 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2114.codfw.wmnet with reason: Maintenance [production]
16:56 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2114.codfw.wmnet with reason: Maintenance [production]
16:50 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2114.codfw.wmnet with reason: Maintenance [production]
16:50 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2114.codfw.wmnet with reason: Maintenance [production]
16:48 <jdrewniak@deploy1002> Synchronized portals: Wikimedia Portals Update: [[gerrit:882682| Bumping portals to master (T128546)]] (duration: 06m 48s) [production]
16:42 <jdrewniak@deploy1002> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:882682| Bumping portals to master (T128546)]] (duration: 06m 48s) [production]
16:41 <otto@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/services/flink-app-example: apply [production]
16:41 <otto@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/services/flink-app-example: apply [production]
16:40 <otto@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
16:40 <otto@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
16:35 <otto@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
16:35 <otto@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
16:32 <marostegui@cumin1001> dbctl commit (dc=all): 'db1170:3317 (re)pooling @ 100%: After adding a column', diff saved to https://phabricator.wikimedia.org/P43261 and previous config saved to /var/cache/conftool/dbconfig/20230123-163207-root.json [production]
16:31 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 100%: After adding a column', diff saved to https://phabricator.wikimedia.org/P43260 and previous config saved to /var/cache/conftool/dbconfig/20230123-163138-root.json [production]
16:17 <marostegui@cumin1001> dbctl commit (dc=all): 'db1170:3317 (re)pooling @ 75%: After adding a column', diff saved to https://phabricator.wikimedia.org/P43259 and previous config saved to /var/cache/conftool/dbconfig/20230123-161702-root.json [production]
16:16 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 75%: After adding a column', diff saved to https://phabricator.wikimedia.org/P43258 and previous config saved to /var/cache/conftool/dbconfig/20230123-161633-root.json [production]
16:01 <marostegui@cumin1001> dbctl commit (dc=all): 'db1170:3317 (re)pooling @ 50%: After adding a column', diff saved to https://phabricator.wikimedia.org/P43257 and previous config saved to /var/cache/conftool/dbconfig/20230123-160157-root.json [production]
16:01 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 50%: After adding a column', diff saved to https://phabricator.wikimedia.org/P43256 and previous config saved to /var/cache/conftool/dbconfig/20230123-160126-root.json [production]
15:53 <sukhe> reprepro -C main include bullseye-wikimedia varnish_6.0.11-1wm1_amd64.changes: T326634 [production]
15:50 <urbanecm> Deploy security patch for T327613 [production]
15:48 <elukey@deploy1002> helmfile [staging] DONE helmfile.d/services/changeprop: sync [production]
15:48 <elukey@deploy1002> helmfile [staging] START helmfile.d/services/changeprop: sync [production]
15:46 <marostegui@cumin1001> dbctl commit (dc=all): 'db1170:3317 (re)pooling @ 25%: After adding a column', diff saved to https://phabricator.wikimedia.org/P43255 and previous config saved to /var/cache/conftool/dbconfig/20230123-154652-root.json [production]
15:46 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 25%: After adding a column', diff saved to https://phabricator.wikimedia.org/P43254 and previous config saved to /var/cache/conftool/dbconfig/20230123-154621-root.json [production]
15:44 <papaul> on going maintenance on fasw-codfw [production]
15:31 <marostegui@cumin1001> dbctl commit (dc=all): 'db1170:3317 (re)pooling @ 10%: After adding a column', diff saved to https://phabricator.wikimedia.org/P43253 and previous config saved to /var/cache/conftool/dbconfig/20230123-153147-root.json [production]
15:31 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 10%: After adding a column', diff saved to https://phabricator.wikimedia.org/P43252 and previous config saved to /var/cache/conftool/dbconfig/20230123-153116-root.json [production]
15:17 <sukhe> reprepro -C main include bullseye-wikimedia trafficserver_9.1.4-1wm1_amd64.changes: T325563 [production]
15:16 <marostegui@cumin1001> dbctl commit (dc=all): 'db1170:3317 (re)pooling @ 5%: After adding a column', diff saved to https://phabricator.wikimedia.org/P43251 and previous config saved to /var/cache/conftool/dbconfig/20230123-151642-root.json [production]
15:16 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 5%: After adding a column', diff saved to https://phabricator.wikimedia.org/P43250 and previous config saved to /var/cache/conftool/dbconfig/20230123-151611-root.json [production]
15:09 <taavi@deploy1002> Finished scap: Backport for [[gerrit:882661|Revert "Enable Linter write namespace tag and template using core config"]] (duration: 07m 28s) [production]
15:03 <taavi@deploy1002> taavi and trainbranchbot: Backport for [[gerrit:882661|Revert "Enable Linter write namespace tag and template using core config"]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet [production]
15:02 <taavi@deploy1002> Started scap: Backport for [[gerrit:882661|Revert "Enable Linter write namespace tag and template using core config"]] [production]
15:01 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1170:3317', diff saved to https://phabricator.wikimedia.org/P43248 and previous config saved to /var/cache/conftool/dbconfig/20230123-150110-marostegui.json [production]
15:00 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1105:3312', diff saved to https://phabricator.wikimedia.org/P43247 and previous config saved to /var/cache/conftool/dbconfig/20230123-150018-marostegui.json [production]
15:00 <taavi@deploy1002> Finished scap: Backport for [[gerrit:880989|Enable Linter write namespace tag and template using core config (T299612)]] (duration: 07m 56s) [production]
14:59 <elukey@deploy1002> helmfile [staging] DONE helmfile.d/services/changeprop: sync [production]
14:59 <elukey@deploy1002> helmfile [staging] START helmfile.d/services/changeprop: sync [production]
14:53 <taavi@deploy1002> taavi and sbailey: Backport for [[gerrit:880989|Enable Linter write namespace tag and template using core config (T299612)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet [production]
14:52 <taavi@deploy1002> Started scap: Backport for [[gerrit:880989|Enable Linter write namespace tag and template using core config (T299612)]] [production]
14:46 <taavi@deploy1002> Finished scap: Backport for [[gerrit:882179|SpecialUserrights: Allow updating the expiry of user groups (T327605)]] (duration: 08m 48s) [production]
14:42 <sukhe> rolling out pybal 1.15.10: T321191 [production]
14:39 <taavi@deploy1002> taavi and func: Backport for [[gerrit:882179|SpecialUserrights: Allow updating the expiry of user groups (T327605)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet [production]
14:37 <taavi@deploy1002> Started scap: Backport for [[gerrit:882179|SpecialUserrights: Allow updating the expiry of user groups (T327605)]] [production]
14:37 <taavi@deploy1002> Finished scap: Backport for [[gerrit:876196|zhwiki: Install PageAssessments (T326387)]] (duration: 11m 24s) [production]
14:27 <taavi@deploy1002> stang and taavi: Backport for [[gerrit:876196|zhwiki: Install PageAssessments (T326387)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet [production]
14:26 <otto@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/services/flink-app-example: apply [production]
14:26 <otto@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/services/flink-app-example: apply [production]
14:25 <taavi@deploy1002> Started scap: Backport for [[gerrit:876196|zhwiki: Install PageAssessments (T326387)]] [production]