4601-4650 of 10000 results (65ms)
2019-05-23 §
04:57 <mobrovac> decommission restbase1009-a - T223976 [production]
04:46 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1080 (duration: 00m 55s) [production]
04:40 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1080 (duration: 00m 58s) [production]
04:24 <mobrovac> start nl, pt, pl wiki dumps to fill the new parsoid tables - T215956 [production]
03:50 <twentyafterfour> m3 database activity levels look like they have returned to normal [production]
03:48 <twentyafterfour> puppet runs cleanly on phab1003 [production]
03:39 <mutante> phab1003 - disabling puppet; /etc/php/7.2/fpm/conf.d# ln -s /etc/php/7.2/mods-available/ldap.ini 20-ldap.ini ; systemctl restart php7.2-fpm [production]
03:27 <twentyafterfour> restarted php-fpm on phab1003 [production]
02:56 <mutante> phab1001 - removing community_metrics and project_changes cron jobs to avoid duplicate mails [production]
02:51 <mutante> phab1003 - chown -R phd /srv/repos/ [production]
02:41 <twentyafterfour> downtimed the systemd state on phab1001 for 1 year [production]
02:35 <mutante> phabricator - going read-write again [production]
02:24 <twentyafterfour> manually started aphlict on phab1003 [production]
02:06 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=phab1003-vcs.eqiad.wmnet [production]
02:04 <mutante> puppetmaster1001 - sudo -i conftool-merge [production]
01:52 <twentyafterfour> phabricator is now served by phab1003 though still in read-only mode for a bit longer [production]
01:52 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=phab1003-vcs.eqiad.wmnet [production]
01:49 <mutante> puppetmaster1001 - conftool-merge [production]
01:41 <eileen> civicrm revision changed from e6e846708f to 21afd001b6, config revision is 87e78d3eac [production]
01:37 <mutante> depooled phab1001-vcs from git-ssh via conftool [production]
01:36 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=phab1001-vcs.eqiad.wmnet [production]
01:33 <mutante> run puppet on mx1001/mx2001 - switch mail route for phab to phab1003 [production]
01:30 <mutante> switched from phab1001 to phab1003 - applied on cp1008 varnish canary first [production]
01:28 <twentyafterfour> stopping phd on phab1001 [production]
01:18 <mutante> phabricator going readonly momentarily [production]
01:09 <twentyafterfour> extended phab downtime in icinga, actual downtime hasn't started yet, prep work taking longer than expected [production]
00:52 <smalyshev@deploy1001> Finished deploy [wdqs/wdqs@e040c6c]: Deploy GUI update (duration: 09m 54s) [production]
00:45 <mutante> phab1003 - rsyncing /srv/repos from phab1001 [production]
00:42 <smalyshev@deploy1001> Started deploy [wdqs/wdqs@e040c6c]: Deploy GUI update [production]
00:33 <ejegg> updated payments-wiki from fa005a0640 to 786d76e212 [production]
2019-05-22 §
23:30 <twentyafterfour> scheduling downtime for phabricator from 0:00 to 1:00 utc [production]
23:10 <maxsem@deploy1001> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/511889/ (duration: 00m 55s) [production]
22:18 <mdholloway> mobileapps rolled back deployment (again) due to occasional references endpoint timeouts [production]
22:17 <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@39e0ef1]: Update mobileapps to fcf3724, take 2 (duration: 07m 19s) [production]
22:15 <foks> reset user email and password for Nv8200pa [production]
22:09 <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@39e0ef1]: Update mobileapps to fcf3724, take 2 [production]
22:09 <mdholloway> mobileapps rolled back deployment due to endpoint check failure (not the same one as before); retrying momentarily [production]
22:08 <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@39e0ef1]: Update mobileapps to fcf3724 (duration: 03m 25s) [production]
22:08 <foks> reset user email and password for DarkKyoushu [production]
22:05 <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@39e0ef1]: Update mobileapps to fcf3724 [production]
21:51 <krinkle@deploy1001> Synchronized php-1.34.0-wmf.5/includes/resourceloader/MessageBlobStore.php: T222539 / 734b3d84f7 (duration: 00m 56s) [production]
21:47 <krinkle@deploy1001> Synchronized php-1.34.0-wmf.6/includes/resourceloader/MessageBlobStore.php: T222539 / 3cb01cc73ce9 (duration: 00m 56s) [production]
21:41 <urandom> decommissioning restbase1008-c -- T223976 [production]
20:46 <mdholloway> mobileapps rolled back deployment due to endpoint check failures [production]
20:43 <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@07632e1]: Update mobileapps to b058298, take 2 (duration: 04m 19s) [production]
20:39 <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@07632e1]: Update mobileapps to b058298, take 2 [production]
20:38 <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@07632e1]: Update mobileapps to b058298 (duration: 02m 41s) [production]
20:35 <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@07632e1]: Update mobileapps to b058298 [production]
19:26 <jforrester@deploy1001> Finished scap: Re-build i18n and re-scap everything for i18n issues for T224116 T224124 T220731 (duration: 32m 55s) [production]
18:53 <jforrester@deploy1001> Started scap: Re-build i18n and re-scap everything for i18n issues for T224116 T224124 T220731 [production]