2301-2350 of 10000 results (29ms)
2020-10-26 §
11:02 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
10:51 <vgutierrez> manually reloading nginx on cloudelastic[1005-1006] [production]
10:29 <vgutierrez> upload trafficserver 8.0.8-1wm3 to apt.wm.org (buster) - T265911 [production]
10:18 <godog> roll restart pybal to apply latest configuration [production]
09:51 <jayme> published docker-registry.discovery.wmnet/eventrouter:0.3.0-3 [production]
09:31 <moritzm> restarting PHP FPM on mw canaries to pick up freetype update [production]
09:04 <godog> swift codfw-prod: bump object weight for ms-be2057 - T261633 [production]
08:58 <moritzm> installing freetype security updates for stretch [production]
08:57 <XioNoX> remove down sessions to AS38758 [production]
08:51 <filippo@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
08:51 <filippo@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:43 <XioNoX> remove down sessions to AS8560 [production]
08:41 <XioNoX> remove down sessions to AS31334 [production]
08:28 <XioNoX> remove down sessions to AS6327 [production]
08:27 <XioNoX> remove down sessions to AS8674 [production]
08:25 <XioNoX> remove down sessions to AS24429 [production]
08:21 <XioNoX> remove down sessions to AS16509 [production]
06:59 <_joe_> rolling restart of php7.2-fpm on the codfw jobrunners, to reduce the number of dangling transcodes after restarting cp-jobqueue for a deploy [production]
06:59 <oblivian@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . [production]
06:16 <oblivian@cumin2001> conftool action : set/pooled=no; selector: cluster=jobrunner,dc=codfw,name=mw224.* [production]
06:15 <oblivian@cumin2001> conftool action : set/pooled=no; selector: cluster=videoscaler,dc=codfw,name=mw228.* [production]
06:10 <marostegui> Warm up tables T261914 [production]
2020-10-25 §
15:53 <dwisehaupt> kernel upgrade and reboot for frdb1003 [production]
15:50 <dwisehaupt> kernel upgrade and reboot for fran1001 [production]
2020-10-23 §
22:56 <mutante> added Nuria to "nda" LDAP group - leaving her in "wmf" until the actual last day - shell account remains so no puppet change needed in ldap_only_admins (T266086) [production]
15:42 <pt1979@cumin2001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:37 <pt1979@cumin2001> START - Cookbook sre.dns.netbox [production]
13:04 <ema> rolling thumbor-instances restart to apply https://gerrit.wikimedia.org/r/c/operations/puppet/+/636012/ T266155 [production]
12:47 <jayme@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'kube-system' for release 'eventrouter' . [production]
10:57 <kormat> uploaded orchestrator v3.2.3 to apt.wikimedia.org buster-wikimedia - T266023 (forgot to log this earlier) [production]
10:56 <volans> uploaded python3-wmflib_0.0.3 to apt.wikimedia.org buster-wikimedia - T257905 [production]
10:09 <jayme> published docker-registry.discovery.wmnet/eventrouter:0.3.0-2 [production]
09:51 <moritzm> masking slapd on the old Stretch replicas to uncover potential direct access outside of the LVSes T264388 [production]
09:47 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:47 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
09:47 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:47 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
09:32 <jayme@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . [production]
09:31 <jayme> published docker-registry.discovery.wmnet/eventrouter:0.3.0-1 [production]
09:26 <jayme@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . [production]
09:23 <jayme@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'wikifeeds' for release 'staging' . [production]
09:09 <volans> upgrading spicerack to 0.0.44 on cumin hosts - T257905 [production]
2020-10-22 §
22:42 <mutante> ganeti1001 - adding 2 more vcpus to VM testreduce1001 - T257940 [production]
22:03 <mutante> deploy1002 - armed keyholder, all deployment keys loaded T265963 [production]
21:56 <mutante> deploy1002 - scap pull and added to mediawiki-installation "dsh" group - will be part of scap trains but just like any appserver (T265963) [production]
20:36 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
20:36 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
19:13 <mutante> deploy1002 currently cloning ALL the deployment repos - new setup [production]
18:57 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
18:57 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]