6151-6200 of 10000 results (86ms)
2022-11-16 ยง
18:33 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175 (T323214)', diff saved to https://phabricator.wikimedia.org/P39982 and previous config saved to /var/cache/conftool/dbconfig/20221116-183336-ladsgroup.json [production]
18:20 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156 (T323214)', diff saved to https://phabricator.wikimedia.org/P39981 and previous config saved to /var/cache/conftool/dbconfig/20221116-182059-ladsgroup.json [production]
18:15 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1175 (T323214)', diff saved to https://phabricator.wikimedia.org/P39980 and previous config saved to /var/cache/conftool/dbconfig/20221116-181505-ladsgroup.json [production]
18:14 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1175.eqiad.wmnet with reason: Maintenance [production]
18:14 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db1175.eqiad.wmnet with reason: Maintenance [production]
18:14 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T323214)', diff saved to https://phabricator.wikimedia.org/P39979 and previous config saved to /var/cache/conftool/dbconfig/20221116-181443-ladsgroup.json [production]
18:10 <urbanecm> Run `time mwscript extensions/GrowthExperiments/maintenance/updateIsActiveFlagForMentees.php --wiki=frwiki` at mwmaint1002 (T318457) [production]
17:59 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P39978 and previous config saved to /var/cache/conftool/dbconfig/20221116-175937-ladsgroup.json [production]
17:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2156 (T323214)', diff saved to https://phabricator.wikimedia.org/P39977 and previous config saved to /var/cache/conftool/dbconfig/20221116-175511-ladsgroup.json [production]
17:55 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2094.codfw.wmnet with reason: Maintenance [production]
17:54 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2094.codfw.wmnet with reason: Maintenance [production]
17:54 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2156.codfw.wmnet with reason: Maintenance [production]
17:54 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db2156.codfw.wmnet with reason: Maintenance [production]
17:54 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149 (T323214)', diff saved to https://phabricator.wikimedia.org/P39976 and previous config saved to /var/cache/conftool/dbconfig/20221116-175434-ladsgroup.json [production]
17:53 <sukhe> rolling restart of varnish to pick up changes in T322903 [production]
17:44 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P39975 and previous config saved to /var/cache/conftool/dbconfig/20221116-174430-ladsgroup.json [production]
17:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P39973 and previous config saved to /var/cache/conftool/dbconfig/20221116-173928-ladsgroup.json [production]
17:29 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T323214)', diff saved to https://phabricator.wikimedia.org/P39972 and previous config saved to /var/cache/conftool/dbconfig/20221116-172924-ladsgroup.json [production]
17:27 <hnowlan@cumin1001> START - Cookbook sre.postgresql.postgres-init [production]
17:26 <hnowlan@cumin1001> END (FAIL) - Cookbook sre.postgresql.postgres-init (exit_code=99) [production]
17:26 <hnowlan> resyncing maps2008 postgres [production]
17:26 <hnowlan@cumin1001> START - Cookbook sre.postgresql.postgres-init [production]
17:26 <hnowlan@puppetmaster1001> conftool action : set/pooled=no; selector: name=maps2008.codfw.wmnet [production]
17:24 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P39971 and previous config saved to /var/cache/conftool/dbconfig/20221116-172421-ladsgroup.json [production]
17:13 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1166 (T323214)', diff saved to https://phabricator.wikimedia.org/P39970 and previous config saved to /var/cache/conftool/dbconfig/20221116-171316-ladsgroup.json [production]
17:13 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1166.eqiad.wmnet with reason: Maintenance [production]
17:13 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db1166.eqiad.wmnet with reason: Maintenance [production]
17:13 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1157 (T323214)', diff saved to https://phabricator.wikimedia.org/P39969 and previous config saved to /var/cache/conftool/dbconfig/20221116-171306-ladsgroup.json [production]
17:10 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T321130)', diff saved to https://phabricator.wikimedia.org/P39968 and previous config saved to /var/cache/conftool/dbconfig/20221116-171003-marostegui.json [production]
17:09 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149 (T323214)', diff saved to https://phabricator.wikimedia.org/P39967 and previous config saved to /var/cache/conftool/dbconfig/20221116-170915-ladsgroup.json [production]
17:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1169 (T318605)', diff saved to https://phabricator.wikimedia.org/P39966 and previous config saved to /var/cache/conftool/dbconfig/20221116-170749-ladsgroup.json [production]
17:07 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1169.eqiad.wmnet with reason: Maintenance [production]
17:07 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1169.eqiad.wmnet with reason: Maintenance [production]
17:00 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2176 (T318605)', diff saved to https://phabricator.wikimedia.org/P39965 and previous config saved to /var/cache/conftool/dbconfig/20221116-170048-ladsgroup.json [production]
16:58 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1157', diff saved to https://phabricator.wikimedia.org/P39964 and previous config saved to /var/cache/conftool/dbconfig/20221116-165759-ladsgroup.json [production]
16:54 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P39963 and previous config saved to /var/cache/conftool/dbconfig/20221116-165457-marostegui.json [production]
16:45 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2176', diff saved to https://phabricator.wikimedia.org/P39962 and previous config saved to /var/cache/conftool/dbconfig/20221116-164542-ladsgroup.json [production]
16:42 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1157', diff saved to https://phabricator.wikimedia.org/P39961 and previous config saved to /var/cache/conftool/dbconfig/20221116-164253-ladsgroup.json [production]
16:39 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P39960 and previous config saved to /var/cache/conftool/dbconfig/20221116-163951-marostegui.json [production]
16:35 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2149 (T323214)', diff saved to https://phabricator.wikimedia.org/P39959 and previous config saved to /var/cache/conftool/dbconfig/20221116-163531-ladsgroup.json [production]
16:35 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2149.codfw.wmnet with reason: Maintenance [production]
16:35 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db2149.codfw.wmnet with reason: Maintenance [production]
16:30 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2176', diff saved to https://phabricator.wikimedia.org/P39958 and previous config saved to /var/cache/conftool/dbconfig/20221116-163035-ladsgroup.json [production]
16:27 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1157 (T323214)', diff saved to https://phabricator.wikimedia.org/P39957 and previous config saved to /var/cache/conftool/dbconfig/20221116-162746-ladsgroup.json [production]
16:24 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T321130)', diff saved to https://phabricator.wikimedia.org/P39956 and previous config saved to /var/cache/conftool/dbconfig/20221116-162444-marostegui.json [production]
16:15 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2176 (T318605)', diff saved to https://phabricator.wikimedia.org/P39955 and previous config saved to /var/cache/conftool/dbconfig/20221116-161529-ladsgroup.json [production]
16:15 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1166 (T321130)', diff saved to https://phabricator.wikimedia.org/P39954 and previous config saved to /var/cache/conftool/dbconfig/20221116-161522-marostegui.json [production]
16:15 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1166.eqiad.wmnet with reason: Maintenance [production]
16:15 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1166.eqiad.wmnet with reason: Maintenance [production]
16:11 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1157 (T323214)', diff saved to https://phabricator.wikimedia.org/P39953 and previous config saved to /var/cache/conftool/dbconfig/20221116-161132-ladsgroup.json [production]