1351-1400 of 10000 results (38ms)
2021-06-16 §
12:08 <marostegui@cumin1001> dbctl commit (dc=all): 'db1131 (re)pooling @ 25%: Repool db1131 after schema change', diff saved to https://phabricator.wikimedia.org/P16541 and previous config saved to /var/cache/conftool/dbconfig/20210616-120818-root.json [production]
12:01 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3:00:00 on maps1007.eqiad.wmnet with reason: Reparenting from maps1009 [production]
12:00 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime for 3:00:00 on maps1007.eqiad.wmnet with reason: Reparenting from maps1009 [production]
12:00 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1131', diff saved to https://phabricator.wikimedia.org/P16540 and previous config saved to /var/cache/conftool/dbconfig/20210616-120015-marostegui.json [production]
11:21 <marostegui@cumin1001> dbctl commit (dc=all): 'db1096:3316 (re)pooling @ 100%: Repool db1096:3316 after schema change', diff saved to https://phabricator.wikimedia.org/P16539 and previous config saved to /var/cache/conftool/dbconfig/20210616-112115-root.json [production]
11:20 <hnowlan> running `nodetool cleanup` on maps1005 [production]
11:06 <marostegui@cumin1001> dbctl commit (dc=all): 'db1096:3316 (re)pooling @ 75%: Repool db1096:3316 after schema change', diff saved to https://phabricator.wikimedia.org/P16538 and previous config saved to /var/cache/conftool/dbconfig/20210616-110612-root.json [production]
10:51 <marostegui@cumin1001> dbctl commit (dc=all): 'db1096:3316 (re)pooling @ 50%: Repool db1096:3316 after schema change', diff saved to https://phabricator.wikimedia.org/P16537 and previous config saved to /var/cache/conftool/dbconfig/20210616-105108-root.json [production]
10:36 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on maps1007.eqiad.wmnet with reason: REIMAGE [production]
10:36 <marostegui@cumin1001> dbctl commit (dc=all): 'db1096:3316 (re)pooling @ 25%: Repool db1096:3316 after schema change', diff saved to https://phabricator.wikimedia.org/P16536 and previous config saved to /var/cache/conftool/dbconfig/20210616-103604-root.json [production]
10:34 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on maps1007.eqiad.wmnet with reason: REIMAGE [production]
10:23 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1096:3316', diff saved to https://phabricator.wikimedia.org/P16535 and previous config saved to /var/cache/conftool/dbconfig/20210616-102349-marostegui.json [production]
09:52 <hnowlan@puppetmaster1001> conftool action : set/pooled=no; selector: name=maps1007.eqiad.wmnet [production]
09:51 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on maps1007.eqiad.wmnet with reason: Reparenting from maps1009 [production]
09:51 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime for 3 days, 0:00:00 on maps1007.eqiad.wmnet with reason: Reparenting from maps1009 [production]
09:50 <hnowlan> disabling puppet on maps1* to reparent maps1007 from new master maps1009 [production]
09:47 <kormat> truncating all pc* tables on pc1010 T282761 [production]
09:40 <kormat@deploy1002> Synchronized wmf-config/db-eqiad.php: Repool pc1009 as pc3 primary T282761 (duration: 00m 59s) [production]
09:04 <kormat> Deploying wmfmariadbpy 0.7.1 T284819 [production]
09:04 <kormat> uploaded wmfmariadbpy 0.7.1 to apt.wm.o [production]
08:24 <Amir1> running "update flaggedrevs set fr_quality = 0 where fr_quality != 0;" on all wikis where flagged revs is enabled (T279761) [production]
07:27 <dcausse> cleanup old /var/log/airflow/scheduler logs to reclaim space on an-airflow1001 [production]
06:55 <volans@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
06:52 <volans@cumin1001> START - Cookbook sre.dns.netbox [production]
05:06 <marostegui> Upgrade clouddb1014 [production]
2021-06-15 §
17:54 <dancy> testing upcoming Scap release on beta [production]
17:21 <mutante> new Wikimedia language "shi" added - Shilha /ˈʃɪlhə/ is a Berber language native to Shilha people. The endonym is Taclḥit /taʃlʜijt/, and in recent English publications the language is often rendered Tashelhiyt or Tashelhit. [production]
17:17 <mutante> new Wikimedia language "dag" added - Dagbani (or Dagbane), also known as Dagbanli and Dagbanle, is a Gur language spoken in Ghana. [production]
17:11 <razzi@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-master1002.eqiad.wmnet with reason: REIMAGE [production]
17:09 <razzi@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on an-master1002.eqiad.wmnet with reason: REIMAGE [production]
16:11 <razzi@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 60 days, 0:00:00 on an-master1002.eqiad.wmnet with reason: Update operating system to bullseye [production]
16:11 <razzi@cumin1001> START - Cookbook sre.hosts.downtime for 60 days, 0:00:00 on an-master1002.eqiad.wmnet with reason: Update operating system to bullseye [production]
14:55 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
14:51 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
14:25 <XioNoX> re-enable cr1-codfw:xe-5/1/2 [production]
13:23 <marostegui> Upgrade clouddb1018 [production]
13:15 <effie> enable puppet on canaries [production]
13:10 <effie> disable puppet on canaries to deploy 699908 [production]
10:45 <XioNoX> re-enable cr1-codfw:xe-5/1/2 [production]
09:42 <XioNoX> cr1-codfw# set interfaces xe-5/1/2 disable [production]
09:25 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db2080', diff saved to https://phabricator.wikimedia.org/P16533 and previous config saved to /var/cache/conftool/dbconfig/20210615-092511-marostegui.json [production]
09:24 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db2086:3318, db2082', diff saved to https://phabricator.wikimedia.org/P16532 and previous config saved to /var/cache/conftool/dbconfig/20210615-092409-marostegui.json [production]
09:08 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2086:3318', diff saved to https://phabricator.wikimedia.org/P16531 and previous config saved to /var/cache/conftool/dbconfig/20210615-090802-marostegui.json [production]
09:06 <marostegui@cumin1001> dbctl commit (dc=all): 'Pool db2083', diff saved to https://phabricator.wikimedia.org/P16530 and previous config saved to /var/cache/conftool/dbconfig/20210615-090650-marostegui.json [production]
09:02 <marostegui@cumin1001> dbctl commit (dc=all): 'Pool db2084', diff saved to https://phabricator.wikimedia.org/P16529 and previous config saved to /var/cache/conftool/dbconfig/20210615-090243-marostegui.json [production]
09:02 <marostegui@cumin1001> dbctl commit (dc=all): 'Pool db2081', diff saved to https://phabricator.wikimedia.org/P16528 and previous config saved to /var/cache/conftool/dbconfig/20210615-090206-marostegui.json [production]
08:59 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2082', diff saved to https://phabricator.wikimedia.org/P16527 and previous config saved to /var/cache/conftool/dbconfig/20210615-085953-marostegui.json [production]
08:59 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db2091', diff saved to https://phabricator.wikimedia.org/P16526 and previous config saved to /var/cache/conftool/dbconfig/20210615-085938-marostegui.json [production]
08:32 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2080 db2083 db2084 db2091', diff saved to https://phabricator.wikimedia.org/P16525 and previous config saved to /var/cache/conftool/dbconfig/20210615-083233-marostegui.json [production]
08:28 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2081', diff saved to https://phabricator.wikimedia.org/P16524 and previous config saved to /var/cache/conftool/dbconfig/20210615-082857-marostegui.json [production]