1851-1900 of 10000 results (40ms)
2021-05-21 §
12:24 <jayme@cumin1001> conftool action : set/pooled=yes; selector: dc=codfw,cluster=docker-registry [production]
12:23 <jayme@cumin1001> conftool action : set/weight=10; selector: dc=codfw,cluster=docker-registry [production]
12:23 <akosiaris@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'mathoid' for release 'staging' . [production]
12:22 <marostegui@cumin1001> dbctl commit (dc=all): 'db1134 (re)pooling @ 100%: Repool db1134', diff saved to https://phabricator.wikimedia.org/P16150 and previous config saved to /var/cache/conftool/dbconfig/20210521-122253-root.json [production]
12:15 <topranks> "Removing BGP peering sessions to LinkedIn AS14413 at AMS-IX / cr2-esams as they are no longer on the exchange." [production]
12:07 <marostegui@cumin1001> dbctl commit (dc=all): 'db1134 (re)pooling @ 75%: Repool db1134', diff saved to https://phabricator.wikimedia.org/P16149 and previous config saved to /var/cache/conftool/dbconfig/20210521-120749-root.json [production]
11:52 <marostegui@cumin1001> dbctl commit (dc=all): 'db1134 (re)pooling @ 50%: Repool db1134', diff saved to https://phabricator.wikimedia.org/P16148 and previous config saved to /var/cache/conftool/dbconfig/20210521-115246-root.json [production]
11:37 <marostegui@cumin1001> dbctl commit (dc=all): 'db1134 (re)pooling @ 25%: Repool db1134', diff saved to https://phabricator.wikimedia.org/P16147 and previous config saved to /var/cache/conftool/dbconfig/20210521-113742-root.json [production]
10:01 <jayme@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host registry2008.codfw.wmnet [production]
09:51 <jayme@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host registry2007.codfw.wmnet [production]
09:41 <jayme@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host registry2006.codfw.wmnet [production]
09:32 <jayme@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host registry2005.codfw.wmnet [production]
09:32 <jayme@cumin1001> START - Cookbook sre.ganeti.makevm for new host registry2008.codfw.wmnet [production]
09:28 <jayme@cumin1001> START - Cookbook sre.ganeti.makevm for new host registry2007.codfw.wmnet [production]
09:26 <gehel> depooling wdqs1006 to catch up on lag [production]
09:24 <jayme@cumin1001> START - Cookbook sre.ganeti.makevm for new host registry2006.codfw.wmnet [production]
09:21 <jayme@cumin1001> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host registry2008.codfw.wmnet [production]
09:21 <jayme@cumin1001> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host registry2007.codfw.wmnet [production]
09:21 <jayme@cumin1001> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host registry2006.codfw.wmnet [production]
09:15 <jayme@cumin1001> START - Cookbook sre.ganeti.makevm for new host registry2008.codfw.wmnet [production]
09:15 <jayme@cumin1001> START - Cookbook sre.ganeti.makevm for new host registry2007.codfw.wmnet [production]
09:15 <jayme@cumin1001> START - Cookbook sre.ganeti.makevm for new host registry2006.codfw.wmnet [production]
09:14 <jayme@cumin1001> START - Cookbook sre.ganeti.makevm for new host registry2005.codfw.wmnet [production]
08:56 <kormat> deploying cumin2002 grants to production T276589 [production]
08:41 <jmm@puppetmaster1001> conftool action : set/pooled=no; selector: name=ldap-replica1002.wikimedia.org [production]
08:41 <jmm@puppetmaster1001> conftool action : set/pooled=no; selector: name=ldap-replica1001.wikimedia.org [production]
08:41 <jmm@puppetmaster1001> conftool action : set/pooled=no; selector: name=ldap-replica2004.wikimedia.org [production]
08:41 <jmm@puppetmaster1001> conftool action : set/pooled=no; selector: name=ldap-replica2003.wikimedia.org [production]
08:20 <marostegui@cumin1001> dbctl commit (dc=all): 'db1119 (re)pooling @ 100%: Repool db1119', diff saved to https://phabricator.wikimedia.org/P16146 and previous config saved to /var/cache/conftool/dbconfig/20210521-082009-root.json [production]
08:05 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1134', diff saved to https://phabricator.wikimedia.org/P16145 and previous config saved to /var/cache/conftool/dbconfig/20210521-080540-marostegui.json [production]
08:05 <marostegui@cumin1001> dbctl commit (dc=all): 'db1119 (re)pooling @ 75%: Repool db1119', diff saved to https://phabricator.wikimedia.org/P16144 and previous config saved to /var/cache/conftool/dbconfig/20210521-080506-root.json [production]
07:50 <marostegui@cumin1001> dbctl commit (dc=all): 'db1119 (re)pooling @ 50%: Repool db1119', diff saved to https://phabricator.wikimedia.org/P16143 and previous config saved to /var/cache/conftool/dbconfig/20210521-075002-root.json [production]
07:34 <marostegui@cumin1001> dbctl commit (dc=all): 'db1119 (re)pooling @ 25%: Repool db1119', diff saved to https://phabricator.wikimedia.org/P16142 and previous config saved to /var/cache/conftool/dbconfig/20210521-073459-root.json [production]
06:32 <moritzm> installing libspring-java security updates on stretch [production]
05:30 <marostegui@cumin1001> dbctl commit (dc=all): 'db1143 (re)pooling @ 100%: Repool db1143', diff saved to https://phabricator.wikimedia.org/P16141 and previous config saved to /var/cache/conftool/dbconfig/20210521-053027-root.json [production]
05:18 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dbstore1006.eqiad.wmnet with reason: REIMAGE [production]
05:16 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on dbstore1006.eqiad.wmnet with reason: REIMAGE [production]
05:15 <marostegui@cumin1001> dbctl commit (dc=all): 'db1143 (re)pooling @ 75%: Repool db1143', diff saved to https://phabricator.wikimedia.org/P16140 and previous config saved to /var/cache/conftool/dbconfig/20210521-051523-root.json [production]
05:14 <moritzm> installing graphviz security updates on stretch [production]
05:00 <marostegui@cumin1001> dbctl commit (dc=all): 'db1143 (re)pooling @ 50%: Repool db1143', diff saved to https://phabricator.wikimedia.org/P16139 and previous config saved to /var/cache/conftool/dbconfig/20210521-050020-root.json [production]
04:59 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db1087.eqiad.wmnet [production]
04:49 <marostegui@cumin1001> START - Cookbook sre.hosts.decommission for hosts db1087.eqiad.wmnet [production]
04:47 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1142', diff saved to https://phabricator.wikimedia.org/P16138 and previous config saved to /var/cache/conftool/dbconfig/20210521-044717-marostegui.json [production]
04:45 <marostegui@cumin1001> dbctl commit (dc=all): 'db1143 (re)pooling @ 25%: Repool db1143', diff saved to https://phabricator.wikimedia.org/P16137 and previous config saved to /var/cache/conftool/dbconfig/20210521-044516-root.json [production]
04:43 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1119', diff saved to https://phabricator.wikimedia.org/P16136 and previous config saved to /var/cache/conftool/dbconfig/20210521-044339-marostegui.json [production]
01:27 <eileen> civicrm revision changed from 35f5afb1b4 to 584b96452a, config revision is 1f8d0a6bfa [production]
01:18 <eileen> civicrm revision changed from 35f5afb1b4 to 584b96452a, config revision is 1f8d0a6bfa [production]
2021-05-20 §
21:45 <herron@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:41 <herron@cumin1001> START - Cookbook sre.dns.netbox [production]
20:30 <ppchelko@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]