1901-1950 of 10000 results (52ms)
2018-03-28 ยง
19:24 <twentyafterfour@tin> Synchronized php: group1 wikis to 1.31.0-wmf.26 (duration: 01m 17s) [production]
19:22 <twentyafterfour@tin> rebuilt and synchronized wikiversions files: group1 wikis to 1.31.0-wmf.26 [production]
19:20 <twentyafterfour> Rolling back to wmf.26 due to increase in fatals: "Replication wait failed: lost connection to MySQL server during query" [production]
19:12 <milimetric@tin> Finished deploy [analytics/refinery@c22fd1e]: Fixing python import bug (duration: 02m 48s) [production]
19:09 <milimetric@tin> Started deploy [analytics/refinery@c22fd1e]: Fixing python import bug [production]
19:09 <milimetric@tin> Started deploy [analytics/refinery@c22fd1e]: (no justification provided) [production]
19:06 <twentyafterfour@tin> Synchronized php: group1 wikis to 1.31.0-wmf.27 (duration: 01m 17s) [production]
19:05 <twentyafterfour@tin> rebuilt and synchronized wikiversions files: group1 wikis to 1.31.0-wmf.27 [production]
19:02 <ebernhardson> restore elasticsearch eqiad disk high/low watermarks to 75/80% with all large reindexes complete [production]
18:52 <urandom> upgrading restbase-dev1005-{a,b} to cassandra 3.11.2 -- T178905 [production]
18:17 <urandom> upgrading restbase-dev1004-b to cassandra 3.11.2 (canary) -- T178905 [production]
18:12 <twentyafterfour@tin> rebuilt and synchronized wikiversions files: group0 wikis to 1.31.0-wmf.27 [production]
18:12 <urandom> upgrading restbase-dev1004-a to cassandra 3.11.2 (canary) -- T178905 [production]
18:03 <twentyafterfour> deploying 1.31.0-wmf.27 to group0. group1 in an hour. See T183966 for blockers. [production]
17:38 <joal@tin> Finished deploy [analytics/refinery@7135d44]: Regular weekly analytics deploy - Scheduled hadoop jobs updates (duration: 05m 21s) [production]
17:32 <joal@tin> Started deploy [analytics/refinery@7135d44]: Regular weekly analytics deploy - Scheduled hadoop jobs updates [production]
16:37 <akosiaris> T189075 upload lttoolbox_3.4.0~r84331-1+wmf1 to apt.wikimedia.org/jessie-wikimedia/main [production]
15:37 <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Enable oversampling for IN, GU, MP in preparation for eqsin (T189252) (duration: 01m 18s) [production]
15:13 <andrewbogott> restarting nodepool on labnodepool1001 (cleanup from T189115) [production]
15:08 <andrewbogott> restarting nova-fullstack on labnet1001 [production]
15:07 <andrewbogott> restarting nova-network on labnet1001 in case it's upset by the rabbit outage [production]
15:02 <andrewbogott> rebooting labservices1001 and labcontrol1001 for T189115 [production]
15:00 <andrewbogott> stopping nova-fullstack on labnet1001 for T189115 [production]
15:00 <andrewbogott> stopping nodepool on labnodepool1001 [production]
14:58 <mobrovac@tin> Synchronized wmf-config/jobqueue.php: Disable redis queue for cirrusSearch jobs for test wikis, file 2/2 - T189137 (duration: 01m 17s) [production]
14:56 <mobrovac@tin> Synchronized wmf-config/InitialiseSettings.php: Disable redis queue for cirrusSearch jobs for test wikis, file 1/2 - T189137 (duration: 01m 17s) [production]
14:54 <ppchelko@tin> Finished deploy [cpjobqueue/deploy@c84880a]: Switch CirrusSearch jobs to kafka for test wikis (duration: 00m 44s) [production]
14:54 <ppchelko@tin> Started deploy [cpjobqueue/deploy@c84880a]: Switch CirrusSearch jobs to kafka for test wikis [production]
13:51 <elukey> reduced number of jobrunner runners on the videoscalers after the last burst of jobs that maxed out the cluster [production]
13:51 <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Enable TemplateStyles on all Wikivoyages (T189838) (duration: 01m 17s) [production]
13:42 <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Enable Wikidata description override on enwik (T184000) (duration: 01m 18s) [production]
13:36 <catrope@tin> Synchronized php-1.31.0-wmf.27/extensions/Echo/modules/nojs/mw.echo.badge.less: Prevent FOUC when loading notification badges (duration: 01m 20s) [production]
13:35 <jynus> upgrade mariadb client on sarin, neodymium, terbium and wasat [production]
13:18 <catrope@tin> Synchronized dblists/flow.dblist: Enable Flow on euwiki (T190500) (duration: 01m 17s) [production]
13:07 <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Enable Translate extension on amwikimedia (T180879) (duration: 01m 22s) [production]
12:35 <twentyafterfour@tin> Finished scap: test running full scap sync from tin (duration: 46m 05s) [production]
11:49 <twentyafterfour@tin> Started scap: test running full scap sync from tin [production]
11:48 <twentyafterfour@tin> Synchronized README: test deploy from tin.eqiad.wmnet (duration: 03m 35s) [production]
10:59 <volans> performing a few minutes live test of reporting Puppet reports to puppetdb too on puppetmaster1001 - T190918 [production]
10:27 <godog> reload icinga on einsteinium after https://gerrit.wikimedia.org/r/c/413142 [production]
10:05 <jynus> upgrade and restart db2093 [production]
09:25 <godog> disable puppet on icinga servers before merging https://gerrit.wikimedia.org/r/c/413142/ [production]
08:25 <arturo> reboot labstore200[2,3,4] for T189115 [production]
08:25 <godog> add more weight to ms-be204[0-3] - T189633 [production]
08:18 <arturo> reboot labstore2001 for T189115 [production]
08:17 <arturo> reboot labstore1002 for T189115 [production]
08:15 <arturo> reboot labstore1001 for T189115 [production]
07:49 <moritzm> uploaded openssl 1.0.2o to apt.wikimedia.org/jessie-wikimedia [production]
06:51 <moritzm> installing remaining ICU security updates [production]
02:28 <l10nupdate@deploy1001> scap sync-l10n completed (1.31.0-wmf.26) (duration: 13m 33s) [production]