1251-1300 of 10000 results (30ms)
2021-01-06 §
16:42 <razzi@cumin1001> END (PASS) - Cookbook sre.hadoop.roll-restart-masters (exit_code=0) [production]
16:32 <jayme@deploy1001> helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. [production]
16:31 <jayme@deploy1001> helmfile [staging-codfw] START helmfile.d/admin 'sync'. [production]
16:15 <jayme@deploy1001> helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. [production]
16:15 <jayme@deploy1001> helmfile [staging-codfw] START helmfile.d/admin 'sync'. [production]
16:02 <razzi@cumin1001> START - Cookbook sre.hadoop.roll-restart-masters [production]
16:01 <moritzm> installing cups security updates on buster (client-side tools/libs) [production]
15:54 <moritzm> installing openexr security updates [production]
15:28 <jayme> imported calico 3.17.1-1 to component/calico-future stretch-wikimedia [production]
15:20 <moritzm> restarting FPM/Apache on mw canaries to pick up p11-kit update [production]
15:04 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
15:02 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
15:01 <moritzm> installing p11-kit security updates on stretch [production]
14:58 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) [production]
14:55 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
13:03 <moritzm> installing tcpdump security updates [production]
12:11 <awight@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:654005|Create rollbacker group on mrwiki (T270864)]] (duration: 01m 21s) [production]
11:33 <moritzm> installing ruby2.5 security updates [production]
11:18 <moritzm> installing libjpeg-turbo security updates on buster [production]
11:14 <moritzm> remove cloudceph2002-dev.wikimedia.org and cloudceph2003-dev.wikimedia.org from debmonitor (got reinstalled as .wmnet) [production]
10:40 <jmm@cumin2001> dbctl commit (dc=all): 'Depool db2140', diff saved to https://phabricator.wikimedia.org/P13658 and previous config saved to /var/cache/conftool/dbconfig/20210106-104029-jmm.json [production]
10:38 <moritzm> depooling db2140 T271084 [production]
08:40 <moritzm> installing Linux 4.9.246 on stretch hosts (no reboots yet) [production]
01:22 <mutante> testreduce1001 rm -rf /srv/deployment/parsoid/deploy [production]
00:55 <eileen> process-control config revision is 9bc3d67b02 [production]
00:46 <eileen> process-control config revision is c38eaa20ed [production]
00:35 <eileen> civicrm revision changed from 6be8a130df to 1d5f6365ba, config revision is d8756a45c1 [production]
2021-01-05 §
23:13 <ladsgroup@deploy1001> Synchronized php-1.36.0-wmf.25/includes/logging/LogPager.php: [[gerrit:654507|Check for the index name while it's being renamed]] (duration: 01m 06s) [production]
22:26 <reedy@deploy1001> Synchronized php-1.36.0-wmf.25/extensions/AbuseFilter/extension.json: T271266 (duration: 01m 04s) [production]
21:48 <jhuneidi@deploy1001> rebuilt and synchronized wikiversions files: Revert "group0 wikis to 1.36.0-wmf.22" [production]
21:12 <razzi@cumin1001> END (PASS) - Cookbook sre.aqs.roll-restart (exit_code=0) [production]
21:02 <razzi@cumin1001> START - Cookbook sre.aqs.roll-restart [production]
20:53 <razzi@deploy1001> Finished deploy [analytics/aqs/deploy@5d05f83]: Configure http request timeout and caching for T268809 (duration: 04m 48s) [production]
20:50 <jhuneidi@deploy1001> rebuilt and synchronized wikiversions files: group0 wikis to 1.36.0-wmf.25 refs T267418 [production]
20:48 <razzi@deploy1001> Started deploy [analytics/aqs/deploy@5d05f83]: Configure http request timeout and caching for T268809 [production]
20:44 <razzi> deploy aqs (analytics query service) as part of analytics train [production]
20:38 <rzl> rzl@mw1362:~$ sudo -i /usr/local/sbin/restart-php7.2-fpm [production]
20:28 <mutante> repooled mw1362 [production]
20:20 <mutante> mw1344 - /usr/local/sbin/restart-php7.2-fpm [production]
20:04 <mutante> mw1344 - restarted apache2 - it was showing the same "partial results" error a mw1362 - no other appservers are showing up in logstash, but these were #1 and #2 source of errors [production]
19:47 <mutante> depooled mw1362 [production]
19:41 <mutante> mw1362 - restarted apache2 [production]
19:29 <razzi@deploy1001> Finished deploy [analytics/refinery@56fb3ff] (thin): Regular analytics weekly train THIN [analytics/refinery@6ce68c950fc339dc3748cf50e6925cd1031287c4] (duration: 00m 08s) [production]
19:29 <razzi@deploy1001> Started deploy [analytics/refinery@56fb3ff] (thin): Regular analytics weekly train THIN [analytics/refinery@6ce68c950fc339dc3748cf50e6925cd1031287c4] [production]
19:28 <razzi@deploy1001> Finished deploy [analytics/refinery@56fb3ff]: Regular analytics weekly train [analytics/refinery@6ce68c950fc339dc3748cf50e6925cd1031287c4] (duration: 09m 37s) [production]
19:19 <razzi@deploy1001> Started deploy [analytics/refinery@56fb3ff]: Regular analytics weekly train [analytics/refinery@6ce68c950fc339dc3748cf50e6925cd1031287c4] [production]
19:17 <razzi> deploying refinery for weekly train [production]
19:16 <mutante> mwdebug1003 - editing apache2 defaults conf and dropping ServerAdmin address.restarting [production]
18:59 <jhuneidi@deploy1001> Finished scap: testwikis wikis to 1.36.0-wmf.25 refs T267418 (duration: 39m 07s) [production]
18:22 <jhuneidi@deploy1001> Started scap: testwikis wikis to 1.36.0-wmf.25 refs T267418 [production]