4651-4700 of 10000 results (37ms)
2021-08-03 ยง
21:37 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
21:10 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
21:06 <ebernhardson@deploy1002> Started deploy [wikimedia/discovery/analytics@2d533ba]: enable glent version marker in final output [production]
21:04 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
20:46 <rzl@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 13 days, 0:00:00 on mw2383.codfw.wmnet with reason: T286463 [production]
20:45 <rzl@cumin1001> START - Cookbook sre.hosts.downtime for 13 days, 0:00:00 on mw2383.codfw.wmnet with reason: T286463 [production]
20:44 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.16/autoload.php: REVERT: 2d4ea752ec6f412ba071ef46023c978d55afcd98: Add (MySQL/DB)PrimaryPos as an alias to (MySQL/DB)MasterPos (T287988; 1/3) (duration: 00m 37s) [production]
20:41 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.16/includes/libs/rdbms/database/position/DBMasterPos.php: REVERT: 2d4ea752ec6f412ba071ef46023c978d55afcd98: Add (MySQL/DB)PrimaryPos as an alias to (MySQL/DB)MasterPos (T287988; 2/3) (duration: 00m 37s) [production]
20:40 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.16/includes/libs/rdbms/database/position/DBMasterPos.php: 2d4ea752ec6f412ba071ef46023c978d55afcd98: Add (MySQL/DB)PrimaryPos as an alias to (MySQL/DB)MasterPos (T287988; 2/3) (duration: 01m 07s) [production]
20:39 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.16/autoload.php: 2d4ea752ec6f412ba071ef46023c978d55afcd98: Add (MySQL/DB)PrimaryPos as an alias to (MySQL/DB)MasterPos (T287988; 1/3) (duration: 01m 08s) [production]
20:23 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on ganeti-test2003.codfw.wmnet with reason: REIMAGE [production]
20:20 <pt1979@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on ganeti-test2003.codfw.wmnet with reason: REIMAGE [production]
20:13 <otto@deploy1002> Finished deploy [analytics/refinery@ea78871] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@ea78871] (duration: 05m 36s) [production]
20:08 <otto@deploy1002> Started deploy [analytics/refinery@ea78871] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@ea78871] [production]
20:08 <otto@deploy1002> Finished deploy [analytics/refinery@ea78871] (thin): Regular analytics weekly train THIN [analytics/refinery@ea78871] (duration: 00m 07s) [production]
20:07 <otto@deploy1002> Started deploy [analytics/refinery@ea78871] (thin): Regular analytics weekly train THIN [analytics/refinery@ea78871] [production]
20:03 <otto@deploy1002> Finished deploy [analytics/refinery@ea78871]: Regular analytics weekly train [analytics/refinery@ea78871] (duration: 20m 38s) [production]
19:57 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
19:52 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
19:45 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on ganeti-test2002.codfw.wmnet with reason: REIMAGE [production]
19:43 <pt1979@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on ganeti-test2002.codfw.wmnet with reason: REIMAGE [production]
19:42 <otto@deploy1002> Started deploy [analytics/refinery@ea78871]: Regular analytics weekly train [analytics/refinery@ea78871] [production]
19:36 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
19:31 <ryankemper> T285355 `ryankemper@an-web1001:~$ sudo run-puppet-agent` to establish `role(analytics_cluster::webserver)` on the host in preparation for upcoming cutover from `thorium`->`an-web1001` [production]
19:31 <otto@deploy1002> Finished deploy [analytics/refinery@aceb561] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@aceb561] (duration: 05m 40s) [production]
19:30 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
19:27 <dduvall@deploy1002> rebuilt and synchronized wikiversions files: revert group0 wikis to 1.37.0-wmf.16 [production]
19:25 <otto@deploy1002> Started deploy [analytics/refinery@aceb561] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@aceb561] [production]
19:14 <dduvall@deploy1002> rebuilt and synchronized wikiversions files: group0 wikis to 1.37.0-wmf.17 [production]
19:01 <otto@deploy1002> Finished deploy [analytics/refinery@aceb561] (thin): Regular analytics weekly train THIN [analytics/refinery@aceb561] (duration: 00m 07s) [production]
19:01 <otto@deploy1002> Started deploy [analytics/refinery@aceb561] (thin): Regular analytics weekly train THIN [analytics/refinery@aceb561] [production]
19:00 <otto@deploy1002> Finished deploy [analytics/refinery@aceb561]: Regular analytics weekly train [analytics/refinery@aceb561] (duration: 16m 25s) [production]
18:47 <Amir1> running mwscript migrateUserGroup.php --wiki=idwiki editor reviewer (T286853) [production]
18:44 <otto@deploy1002> Started deploy [analytics/refinery@aceb561]: Regular analytics weekly train [analytics/refinery@aceb561] [production]
18:29 <dduvall@deploy1002> Finished scap: testwikis wikis to 1.37.0-wmf.17 (duration: 36m 44s) [production]
18:18 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
18:12 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
18:06 <ebernhardson@deploy1002> Finished deploy [search/mjolnir/deploy@f0f70d1]: T286642 fixes to bulk daemon prioritization (duration: 00m 48s) [production]
18:05 <ebernhardson@deploy1002> Started deploy [search/mjolnir/deploy@f0f70d1]: T286642 fixes to bulk daemon prioritization [production]
17:52 <dduvall@deploy1002> Started scap: testwikis wikis to 1.37.0-wmf.17 [production]
16:59 <hashar> Gerrit has been upgraded [production]
16:47 <dancy@deploy1002> Finished deploy [gerrit/gerrit@244120b]: Gerrit to 3.3.5 on gerrit1001 (duration: 00m 07s) [production]
16:47 <dancy@deploy1002> Started deploy [gerrit/gerrit@244120b]: Gerrit to 3.3.5 on gerrit1001 [production]
16:45 <urbanecm> Start server side upload for 1 video file (T287957) [production]
16:45 <hashar> Stopping Gerrit for upgrade [production]
16:43 <volans> upgraded spicerack to 0.0.57-1+deb10u1 on cumin1001 [production]
16:36 <dancy@deploy1002> Finished deploy [gerrit/gerrit@244120b]: Gerrit to 3.3.5 on gerrit2001 (duration: 00m 10s) [production]
16:36 <dancy@deploy1002> Started deploy [gerrit/gerrit@244120b]: Gerrit to 3.3.5 on gerrit2001 [production]
16:27 <hashar> Going to upgrade Gerrit 3.3 (scheduled maintenance) [production]
16:18 <pt1979@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]