9801-9850 of 10000 results (75ms)
2018-03-29 §
09:16 <elukey> roll restart aqs on aqs100* for icu/openssl upgrades [production]
08:18 <akosiaris> T189075 upload apertium_3.5.1-1+wmf1 to apt.wikimedia.org/jessie-wikimedia/main [production]
08:18 <moritzm> installing OpenJDK security updates on elastic* hosts (along with current version of the search plugins package) [production]
08:07 <elukey> roll restart of cassandra on aqs* for openjdk-8 upgrades [production]
07:20 <moritzm> installing openssl security updates [production]
07:18 <ema> reboot cache@eqiad for retpoline kernel updates: T188092 [production]
04:35 <twentyafterfour> ran scap pull on deploy1001 [production]
02:00 <l10nupdate@tin> LocalisationUpdate failed: git pull of core failed [production]
2018-03-28 §
23:50 <eileen> update civicrm revision changed from 9478ca39f1 to d6855cd281 (further security module updates, engage import dedupe) [production]
23:38 <ebernhardson@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: T187148: Configure next Cirrus AB test (duration: 01m 16s) [production]
23:18 <ebernhardson@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: T184969: Enable PageAssessments on trwiki (duration: 01m 09s) [production]
23:13 <MaxSem> created PageAssessments tables on trwiki [production]
22:17 <twentyafterfour@tin> Synchronized php: group1 wikis to 1.31.0-wmf.26 (duration: 01m 18s) [production]
22:16 <twentyafterfour@tin> rebuilt and synchronized wikiversions files: group1 wikis to 1.31.0-wmf.26 [production]
22:13 <twentyafterfour> deploy of 1.31.0-wmf.27 resulted in a lot of SlowTimer errors for SlowTimer [10000ms] at runtime/ext_mysql: slow query: SELECT MASTER_GTID_WAIT(...) [production]
22:12 <eileen> civicrm revision changed from 3f6028b24f to 9478ca39f1 (drupal security update) [production]
22:09 <twentyafterfour@tin> rebuilt and synchronized wikiversions files: sync https://gerrit.wikimedia.org/r/#/c/422563/ group1 wikis to 1.31.0-wmf.27 refs T183966 T190960 [production]
22:08 <twentyafterfour> rolling forward group1 to 1.31.0-wmf.27 refs T183966 T190960 [production]
22:05 <twentyafterfour@tin> Synchronized php-1.31.0-wmf.27/includes/: sync https://gerrit.wikimedia.org/r/#/c/422565/ (duration: 02m 15s) [production]
22:03 <twentyafterfour> syncing https://gerrit.wikimedia.org/r/#/c/422565/ refs T190960 T183966 [production]
21:53 <mutante> deploy1001 - revoking old puppet certs and signing new ones [production]
21:42 <twentyafterfour> getting the train back on track, group1 wikis to 1.31.0-wmf.27 [production]
20:51 <XenoRyet> updated civicrm from 85c89c7d0a to 3f6028b24f [production]
20:50 <bsitzmann@tin> Finished deploy [mobileapps/deploy@6a0d877]: Update mobileapps to a5833a0 (duration: 05m 36s) [production]
20:44 <bsitzmann@tin> Started deploy [mobileapps/deploy@6a0d877]: Update mobileapps to a5833a0 [production]
20:12 <mlitn@tin> Finished deploy [3d2png/deploy@c447488]: Updating 3d2png (duration: 02m 26s) [production]
20:09 <mlitn@tin> Started deploy [3d2png/deploy@c447488]: Updating 3d2png [production]
19:54 <mutante> deploy1001 - schedule downtime for reinstall with jessie, reinstalling (T175288) [production]
19:24 <twentyafterfour@tin> Synchronized php: group1 wikis to 1.31.0-wmf.26 (duration: 01m 17s) [production]
19:22 <twentyafterfour@tin> rebuilt and synchronized wikiversions files: group1 wikis to 1.31.0-wmf.26 [production]
19:20 <twentyafterfour> Rolling back to wmf.26 due to increase in fatals: "Replication wait failed: lost connection to MySQL server during query" [production]
19:12 <milimetric@tin> Finished deploy [analytics/refinery@c22fd1e]: Fixing python import bug (duration: 02m 48s) [production]
19:09 <milimetric@tin> Started deploy [analytics/refinery@c22fd1e]: Fixing python import bug [production]
19:09 <milimetric@tin> Started deploy [analytics/refinery@c22fd1e]: (no justification provided) [production]
19:06 <twentyafterfour@tin> Synchronized php: group1 wikis to 1.31.0-wmf.27 (duration: 01m 17s) [production]
19:05 <twentyafterfour@tin> rebuilt and synchronized wikiversions files: group1 wikis to 1.31.0-wmf.27 [production]
19:02 <ebernhardson> restore elasticsearch eqiad disk high/low watermarks to 75/80% with all large reindexes complete [production]
18:52 <urandom> upgrading restbase-dev1005-{a,b} to cassandra 3.11.2 -- T178905 [production]
18:17 <urandom> upgrading restbase-dev1004-b to cassandra 3.11.2 (canary) -- T178905 [production]
18:12 <twentyafterfour@tin> rebuilt and synchronized wikiversions files: group0 wikis to 1.31.0-wmf.27 [production]
18:12 <urandom> upgrading restbase-dev1004-a to cassandra 3.11.2 (canary) -- T178905 [production]
18:03 <twentyafterfour> deploying 1.31.0-wmf.27 to group0. group1 in an hour. See T183966 for blockers. [production]
17:38 <joal@tin> Finished deploy [analytics/refinery@7135d44]: Regular weekly analytics deploy - Scheduled hadoop jobs updates (duration: 05m 21s) [production]
17:32 <joal@tin> Started deploy [analytics/refinery@7135d44]: Regular weekly analytics deploy - Scheduled hadoop jobs updates [production]
16:37 <akosiaris> T189075 upload lttoolbox_3.4.0~r84331-1+wmf1 to apt.wikimedia.org/jessie-wikimedia/main [production]
15:37 <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Enable oversampling for IN, GU, MP in preparation for eqsin (T189252) (duration: 01m 18s) [production]
15:13 <andrewbogott> restarting nodepool on labnodepool1001 (cleanup from T189115) [production]
15:08 <andrewbogott> restarting nova-fullstack on labnet1001 [production]
15:07 <andrewbogott> restarting nova-network on labnet1001 in case it's upset by the rabbit outage [production]
15:02 <andrewbogott> rebooting labservices1001 and labcontrol1001 for T189115 [production]