1301-1350 of 10000 results (30ms)
2022-01-10 §
12:34 <taavi@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:752634|hewikisource: remove "קטע" namespace and its talk page (T298430)]] (duration: 00m 58s) [production]
12:30 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1146:3312 (T297191)', diff saved to https://phabricator.wikimedia.org/P18464 and previous config saved to /var/cache/conftool/dbconfig/20220110-123009-marostegui.json [production]
12:29 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
12:28 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1146:3312 (T297191)', diff saved to https://phabricator.wikimedia.org/P18463 and previous config saved to /var/cache/conftool/dbconfig/20220110-122847-marostegui.json [production]
12:28 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1146.eqiad.wmnet with reason: Maintenance [production]
12:28 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1146.eqiad.wmnet with reason: Maintenance [production]
12:28 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1170:3312 (T297191)', diff saved to https://phabricator.wikimedia.org/P18462 and previous config saved to /var/cache/conftool/dbconfig/20220110-122840-marostegui.json [production]
12:27 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
12:27 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
12:24 <taavi@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:752187|Growth: Add GEMentorDashboardDeploymentMode (T298792)]] (duration: 00m 59s) [production]
12:24 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
12:19 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
12:18 <taavi@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:751545|uzwiki: Amend Babel configuration (T131924)]] (duration: 00m 59s) [production]
12:14 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
12:14 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
12:13 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1170:3312', diff saved to https://phabricator.wikimedia.org/P18460 and previous config saved to /var/cache/conftool/dbconfig/20220110-121335-marostegui.json [production]
12:10 <taavi@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:747868|Add MediaSearch profiles (T297863)]] (duration: 00m 59s) [production]
12:10 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
11:58 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1170:3312', diff saved to https://phabricator.wikimedia.org/P18459 and previous config saved to /var/cache/conftool/dbconfig/20220110-115830-marostegui.json [production]
11:43 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1170:3312 (T297191)', diff saved to https://phabricator.wikimedia.org/P18458 and previous config saved to /var/cache/conftool/dbconfig/20220110-114326-marostegui.json [production]
11:43 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1170:3312 (T297191)', diff saved to https://phabricator.wikimedia.org/P18457 and previous config saved to /var/cache/conftool/dbconfig/20220110-114305-marostegui.json [production]
11:43 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1170.eqiad.wmnet with reason: Maintenance [production]
11:42 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1170.eqiad.wmnet with reason: Maintenance [production]
11:42 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db[1155-1156].eqiad.wmnet with reason: Maintenance [production]
11:42 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db[1155-1156].eqiad.wmnet with reason: Maintenance [production]
11:42 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1182.eqiad.wmnet with reason: Maintenance [production]
11:41 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1182.eqiad.wmnet with reason: Maintenance [production]
11:41 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1162.eqiad.wmnet with reason: Maintenance [production]
11:41 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1162.eqiad.wmnet with reason: Maintenance [production]
11:41 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
11:41 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
11:40 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on 9 hosts with reason: Maintenance [production]
11:40 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on 9 hosts with reason: Maintenance [production]
11:40 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1105:3312 (T297191)', diff saved to https://phabricator.wikimedia.org/P18456 and previous config saved to /var/cache/conftool/dbconfig/20220110-114043-marostegui.json [production]
11:25 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1105:3312', diff saved to https://phabricator.wikimedia.org/P18455 and previous config saved to /var/cache/conftool/dbconfig/20220110-112538-marostegui.json [production]
11:10 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1105:3312', diff saved to https://phabricator.wikimedia.org/P18454 and previous config saved to /var/cache/conftool/dbconfig/20220110-111034-marostegui.json [production]
10:55 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1105:3312 (T297191)', diff saved to https://phabricator.wikimedia.org/P18453 and previous config saved to /var/cache/conftool/dbconfig/20220110-105529-marostegui.json [production]
10:53 <moritzm> installing openjdk-11 security updates [production]
10:44 <marostegui@cumin1001> dbctl commit (dc=all): 'Remove logpager group from s7 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P18452 and previous config saved to /var/cache/conftool/dbconfig/20220110-104445-marostegui.json [production]
10:40 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1105:3312 (T297191)', diff saved to https://phabricator.wikimedia.org/P18451 and previous config saved to /var/cache/conftool/dbconfig/20220110-104004-marostegui.json [production]
10:40 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1105.eqiad.wmnet with reason: Maintenance [production]
10:39 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1105.eqiad.wmnet with reason: Maintenance [production]
10:39 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
10:39 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
10:39 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1139.eqiad.wmnet with reason: Maintenance [production]
10:39 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1139.eqiad.wmnet with reason: Maintenance [production]
10:39 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
10:39 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
10:38 <elukey> stop/start kafka daemons on kafka-main1* nodes to move the kafka user to fixed uid/gid - T296641 [production]
10:34 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on dbstore1005.eqiad.wmnet with reason: Maintenance [production]