651-700 of 10000 results (30ms)
2021-01-03 §
14:42 <andrewbogott> restarting slapd on serpens and seaborgium [production]
14:11 <milimetric> reset-failed refinery-sqoop-whole-mediawiki.service [analytics]
14:10 <milimetric> manual sqoop finished, logs on an-launcher1002 at /var/log/refinery/sqoop-mediawiki.log and /var/log/refinery/sqoop-mediawiki-production.log [analytics]
11:57 <wm-bot> <lucaswerkmeister> deployed db1e890252 (grab cursor for draggable links) [tools.lexeme-forms]
11:38 <elukey> powercycle an-worker1114 (kernel errors in the serial console) [production]
09:07 <elukey> reboot ms-be2050 as attempt to recover/fix its broken networking state (started from Dec 30th) - T271041 [production]
07:06 <dcaro> Got a network hiccup on cloudnet1004, keeping track here T271058 [admin]
2021-01-02 §
19:27 <vgutierrez> restart acme-chief on acmechief1001 [production]
19:13 <James_F> Zuul: Add CI for the Mirage skin T270979 [releng]
16:18 <Majavah> deploy 59133d0 [tools.majavah-bot]
2021-01-01 §
18:26 <James_F> zuul: Try in a second way to only run mwext coverage jobs on master T270976 [releng]
18:13 <James_F> zuul: [mediawiki/extensions/AbuseFilter] Make sqlite tests voting T251967 [releng]
17:31 <Majavah> deploy 9876262 to fix bug with ip ranges [tools.majavah-bot]
14:54 <milimetric> deployed refinery hotfix for sqoop problem, after testing on three small wikis [analytics]
14:49 <milimetric@deploy1001> Finished deploy [analytics/refinery@f9281dd] (thin): [SAFE, IGNORE] Simple hotfix for a python bug, analytics refinery only, not urgent (duration: 00m 07s) [production]
14:49 <milimetric@deploy1001> Started deploy [analytics/refinery@f9281dd] (thin): [SAFE, IGNORE] Simple hotfix for a python bug, analytics refinery only, not urgent [production]
14:48 <milimetric@deploy1001> Finished deploy [analytics/refinery@f9281dd]: [SAFE, IGNORE] Simple hotfix for a python bug, analytics refinery only, not urgent (duration: 10m 00s) [production]
14:38 <milimetric@deploy1001> Started deploy [analytics/refinery@f9281dd]: [SAFE, IGNORE] Simple hotfix for a python bug, analytics refinery only, not urgent [production]
08:52 <legoktm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Switch fiwiki to their 500k temporary logo! (T270974) (duration: 00m 55s) [production]
08:46 <legoktm@deploy1001> Synchronized static/images/project-logos/fiwiki-500k-2x.png: Add fiwiki 500k temporary logos (3/3) (duration: 00m 55s) [production]
08:45 <legoktm@deploy1001> Synchronized static/images/project-logos/fiwiki-500k-1.5x.png: Add fiwiki 500k temporary logos (2/3) (duration: 00m 54s) [production]
08:44 <legoktm@deploy1001> Synchronized static/images/project-logos/fiwiki-500k.png: Add fiwiki 500k temporary logos (1/3) (duration: 00m 58s) [production]
2020-12-31 §
01:47 <James_F> Docker: Building and publishing tox-homer:0.1.1 [releng]
01:36 <James_F> Docker: Building and publishing tox-eventlogging:0.4.1 [releng]
01:19 <James_F> Docker: Building and publishing tox-conftool:0.4.1 [releng]
01:04 <James_F> Docker: Building and publishing tox-censorshipmonitoring:0.1.1 [releng]
2020-12-30 §
12:22 <wm-bot> <lucaswerkmeister> deployed 191518cbf9 (edit lemma when adding first form) [tools.lexeme-forms]
2020-12-29 §
15:52 <vgutierrez> reloading nginx on cloudelastic1005 and cloudelastic1006 [production]
15:48 <vgutierrez> triggering a puppet run on cp nodes [production]
15:45 <vgutierrez> restarting acme-chief on acmechief1001 [production]
09:18 <elukey> restart hue to pick up analytics-hive endpoint settings [analytics]
2020-12-28 §
12:32 <arturo> stop doing backups for the dumps project https://gerrit.wikimedia.org/r/c/operations/puppet/+/652182 (T260692) [admin]
12:32 <arturo> stop doing backups for the dumps project https://gerrit.wikimedia.org/r/c/operations/puppet/+/652182 (T260682) [admin]
12:23 <arturo> icinga downtime cloudvirt1026 disk space check until january 5 (T260692) [admin]
09:54 <elukey> reboot an-coord1002 (puppet in D state after issues with broken disk - host in standby, no traffic) [production]
09:00 <wm-bot> <lucaswerkmeister> updated Python packages [tools.quickcategories]
06:15 <andrewbogott> restarting designate-central on cloudservices1003/1004. I'm pretty sure they're distressed because of DB lag but it's worth a try [admin]
2020-12-24 §
12:41 <elukey@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
12:37 <elukey@cumin1001> START - Cookbook sre.dns.netbox [production]
12:34 <elukey@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
12:23 <elukey@cumin1001> START - Cookbook sre.dns.netbox [production]
11:22 <volans> running on cumin1001: homer asw2-*-eqiad.mgmt.eqiad.wmnet commit "Fix numbering of an-worker hosts - T260445" [production]
11:08 <hashar> gerrit2001 (replica) restarting Gerrit server [production]
00:45 <legoktm> reset maxmind password [production]
2020-12-23 §
21:33 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:30 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
20:40 <legoktm> deploying https://gerrit.wikimedia.org/r/651819 [releng]
20:32 <bstorm> Created the directory /srv/misc/shared/wikilink/project on labstore1004 and verified puppet and nfs-exportd are happy T264107 [wikilink]
19:48 <James_F> Zuul: [mediawiki/tools/dependency-analysis] Add composer test CI [releng]
19:20 <bstorm> created clouddb-wikireplicas-proxy-1 and clouddb-wikireplicas-proxy-2 as well as the 16 neutron ports for wikireplicas proxying [clouddb-services]