1-50 of 10000 results (20ms)
2021-01-05 ยง
23:13 <ladsgroup@deploy1001> Synchronized php-1.36.0-wmf.25/includes/logging/LogPager.php: [[gerrit:654507|Check for the index name while it's being renamed]] (duration: 01m 06s) [production]
22:26 <reedy@deploy1001> Synchronized php-1.36.0-wmf.25/extensions/AbuseFilter/extension.json: T271266 (duration: 01m 04s) [production]
21:48 <jhuneidi@deploy1001> rebuilt and synchronized wikiversions files: Revert "group0 wikis to 1.36.0-wmf.22" [production]
21:12 <razzi@cumin1001> END (PASS) - Cookbook sre.aqs.roll-restart (exit_code=0) [production]
21:02 <razzi@cumin1001> START - Cookbook sre.aqs.roll-restart [production]
20:53 <razzi@deploy1001> Finished deploy [analytics/aqs/deploy@5d05f83]: Configure http request timeout and caching for T268809 (duration: 04m 48s) [production]
20:50 <jhuneidi@deploy1001> rebuilt and synchronized wikiversions files: group0 wikis to 1.36.0-wmf.25 refs T267418 [production]
20:48 <razzi@deploy1001> Started deploy [analytics/aqs/deploy@5d05f83]: Configure http request timeout and caching for T268809 [production]
20:44 <razzi> deploy aqs (analytics query service) as part of analytics train [production]
20:38 <rzl> rzl@mw1362:~$ sudo -i /usr/local/sbin/restart-php7.2-fpm [production]
20:28 <mutante> repooled mw1362 [production]
20:20 <mutante> mw1344 - /usr/local/sbin/restart-php7.2-fpm [production]
20:04 <mutante> mw1344 - restarted apache2 - it was showing the same "partial results" error a mw1362 - no other appservers are showing up in logstash, but these were #1 and #2 source of errors [production]
19:47 <mutante> depooled mw1362 [production]
19:41 <mutante> mw1362 - restarted apache2 [production]
19:29 <razzi@deploy1001> Finished deploy [analytics/refinery@56fb3ff] (thin): Regular analytics weekly train THIN [analytics/refinery@6ce68c950fc339dc3748cf50e6925cd1031287c4] (duration: 00m 08s) [production]
19:29 <razzi@deploy1001> Started deploy [analytics/refinery@56fb3ff] (thin): Regular analytics weekly train THIN [analytics/refinery@6ce68c950fc339dc3748cf50e6925cd1031287c4] [production]
19:28 <razzi@deploy1001> Finished deploy [analytics/refinery@56fb3ff]: Regular analytics weekly train [analytics/refinery@6ce68c950fc339dc3748cf50e6925cd1031287c4] (duration: 09m 37s) [production]
19:19 <razzi@deploy1001> Started deploy [analytics/refinery@56fb3ff]: Regular analytics weekly train [analytics/refinery@6ce68c950fc339dc3748cf50e6925cd1031287c4] [production]
19:17 <razzi> deploying refinery for weekly train [production]
19:16 <mutante> mwdebug1003 - editing apache2 defaults conf and dropping ServerAdmin address.restarting [production]
18:59 <jhuneidi@deploy1001> Finished scap: testwikis wikis to 1.36.0-wmf.25 refs T267418 (duration: 39m 07s) [production]
18:22 <jhuneidi@deploy1001> Started scap: testwikis wikis to 1.36.0-wmf.25 refs T267418 [production]
18:21 <mbsantos@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'mobileapps' for release 'production' . [production]
18:18 <mbsantos@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'mobileapps' for release 'production' . [production]
18:13 <elukey> run homer on cr1/cr2-eqiad to update the analytics-in4 filter (https://gerrit.wikimedia.org/r/c/operations/homer/public/+/654469) [production]
18:08 <mbsantos@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'mobileapps' for release 'staging' . [production]
17:10 <longma> 1.36.0-wmf.25 was branched at 083fd09afcd204cfef177e11d7a5e4fd1217acfc for T267418 [production]
17:00 <XioNoX> capture packets on pfw3-eqiad:reth0.1134 - T263833 [production]
15:50 <jbond42> merging puppetlabs-lvm update [production]
15:41 <volans> upgraded wmflib to 0.0.6 on all hosts where it's installed - T257905 [production]
15:37 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc2025.codfw.wmnet with reason: REIMAGE [production]
15:35 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mc2025.codfw.wmnet with reason: REIMAGE [production]
15:35 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1025.eqiad.wmnet with reason: REIMAGE [production]
15:33 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mc1025.eqiad.wmnet with reason: REIMAGE [production]
14:59 <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Remove overrides from wgEventLoggingSchemas (duration: 00m 57s) [production]
13:40 <moritzm> installing python-apt security updates on buster/stretch [production]
13:29 <moritzm> installing xen security updates on buster [production]
13:01 <moritzm> installing lxml security updates for stretch [production]
12:48 <elukey> add PXE d-i rescue bootable image config for jessie/stretch/buster to tftp [production]
12:43 <jmm@cumin2001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
12:29 <jmm@cumin2001> START - Cookbook sre.dns.netbox [production]
12:13 <sukhe@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:10:00 on malmok.wikimedia.org with reason: rebooting for kernel update [production]
12:13 <sukhe@cumin1001> START - Cookbook sre.hosts.downtime for 0:10:00 on malmok.wikimedia.org with reason: rebooting for kernel update [production]
12:12 <moritzm> installing p11-kit security updates on buster [production]
12:01 <marostegui> Restart db2121 T271106 [production]
11:53 <moritzm> installing lxml security updates for buster [production]
11:02 <marostegui@cumin1001> dbctl commit (dc=all): 'db1074 (re)pooling @ 100%: After cloning db1155:3312', diff saved to https://phabricator.wikimedia.org/P13656 and previous config saved to /var/cache/conftool/dbconfig/20210105-110246-root.json [production]
10:56 <jmm@cumin2001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
10:49 <jmm@cumin2001> START - Cookbook sre.dns.netbox [production]