4251-4300 of 10000 results (57ms)
2018-02-15 ยง
11:58 <addshore> addshore@terbium:~$ mwscript extensions/Cognate/maintenance/populateCognatePages.php --wiki elwiktionary --batchsize 1000 # T185738 [production]
11:49 <milimetric@tin> Started deploy [analytics/refinery@26d4e50]: Deploying Refinery jobs with new 0.0.58 jars [production]
10:58 <marostegui> Stop replication in sync db1089 and db1066 [production]
10:55 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic db1097:3314 and slowly repool db1097:3315 (duration: 01m 12s) [production]
10:38 <jynus@tin> Synchronized wmf-config/db-codfw.php: Depool db2042 fully (duration: 01m 12s) [production]
10:28 <marostegui> Upgrade mariadb on db1066 [production]
10:19 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic db1097:3314 (duration: 01m 12s) [production]
09:58 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Slowly repool db1097:3314 (duration: 01m 12s) [production]
09:47 <marostegui> Deploy schema change on db1097:3315 - T187089 T185128 T153182 [production]
09:39 <marostegui> Upgrade kernel and mariadb on db1097 [production]
09:39 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1097 for s4 and s5 (duration: 01m 12s) [production]
09:29 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Fully repool db1082 (duration: 01m 12s) [production]
09:07 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic db1082 (duration: 01m 12s) [production]
08:54 <moritzm> installing erlang security updates on labtestcontrol* [production]
08:39 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Slowly repool db1082 (duration: 01m 13s) [production]
08:18 <marostegui> Upgrade kernel + mariadb on db1082 (sanitarium master in s5) [production]
07:55 <marostegui> Stop replication in sync on db1089 and db1066 - T162807 [production]
07:54 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1089 and db1066 - T162807 (duration: 01m 12s) [production]
07:39 <marostegui> Deploy schema change on db1082 (sanitarium master) with replication, this will generate lag on labs - T187089 T185128 T153182 [production]
07:39 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1082 (duration: 01m 13s) [production]
07:35 <moritzm> installing libvorbis security updates on stretch [production]
07:30 <twentyafterfour> phabricator upgrade finished. phd is back online. [production]
07:27 <twentyafterfour> phabricator database migration finished [production]
07:26 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Fully repool db1110 (duration: 01m 12s) [production]
07:09 <jynus> reimage dbproxy1003 to stretch [production]
07:04 <twentyafterfour> Applying patch "phabricator:20180215.maniphest.02.populate.php" to host "m3-master.eqiad.wmnet"... [production]
07:02 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase weight for db1110 (duration: 01m 13s) [production]
06:57 <twentyafterfour> apache restarted, update appears to be successful [production]
06:57 <twentyafterfour> phabricator database migrations applied [production]
06:50 <twentyafterfour> shutting down apache on phab1001 to deploy update, downtime should be only a couple of minutes [production]
06:49 <twentyafterfour> starting phabricator upgrade tagged release/2018-02-15/1 [production]
06:45 <twentyafterfour> restarted apache on phab1001 and reset cluster.read-only to false [production]
06:44 <jynus> set db1059 in read-write [production]
06:38 <jynus> merging dns update for phabricator db [production]
06:35 <jynus> set db1043 as read only [production]
06:34 <twentyafterfour> set cluster.read-only in phabricator [production]
06:33 <jynus> about to set phabricator.wikimedia.org as read only [production]
06:28 <jynus> scheduling downtime for phabricator on phab1001 [production]
06:17 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Slowly repool db1110 (duration: 01m 13s) [production]
06:06 <marostegui> Upgrade mysql on db1110 [production]
05:57 <jynus> restarting dbproxy1008 for kernel upgrade [production]
05:43 <andrew@tin> Finished deploy [horizon/deploy@c355366]: testing a couple of cherry-picks in horizon (duration: 03m 06s) [production]
05:40 <andrew@tin> Started deploy [horizon/deploy@c355366]: testing a couple of cherry-picks in horizon [production]
02:25 <l10nupdate@tin> scap sync-l10n completed (1.31.0-wmf.20) (duration: 07m 25s) [production]
02:01 <mutante> phab1001 - restarted apache to fix server status page [production]
01:27 <twentyafterfour> restarting apache2 on phab1001 to free deadlocked php processes. [production]
01:03 <twentyafterfour> using the current phabricator maintenance window to deploy https://gerrit.wikimedia.org/r/#/c/410626/ [production]
01:03 <twentyafterfour> the scheduled phabricator upgrade is delayed until 06:00 UTC Thursday because of large database migrations. Doing the upgrade at a time when DBAs are available to assist. [production]
00:52 <maxsem@tin> Synchronized wmf-config/: https://gerrit.wikimedia.org/r/#/c/410267/ (duration: 01m 14s) [production]
00:49 <maxsem@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/410267/ (duration: 01m 13s) [production]