3151-3200 of 10000 results (67ms)
2019-03-27 §
08:09 <godog> bounce rsyslog on cobalt - apache access logs stopped at ~6.30 today [production]
08:07 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1122 (duration: 00m 57s) [production]
07:11 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1122 (duration: 00m 58s) [production]
06:57 <SMalyshev> depooled wdqs1005 to catch up [production]
06:56 <SMalyshev> repooled wdqs1004 [production]
06:42 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1103:3312 (duration: 00m 58s) [production]
06:02 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Change one parsercache key on codfw - T210725 (duration: 00m 57s) [production]
05:58 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1103:3312 (duration: 01m 10s) [production]
00:56 <SMalyshev> depooled wdq1004 to catch up [production]
00:55 <SMalyshev> repooled wdq1006 [production]
2019-03-26 §
23:37 <SMalyshev> repooled wdqs2003 [production]
23:12 <ebernhardson@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: T216206 : sync noop labs config: Actually load WBCS-Lexeme extension before trying to use it (duration: 00m 57s) [production]
22:12 <gehel> freezing and unfreezing writes to elasticsearch codfw [production]
21:47 <SMalyshev> depool wdq2003 to catch it up [production]
21:32 <ebernhardson> manually thaw search.svc.codfw.wmnet:9643 [production]
21:31 <mholloway-shell@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable WikimediaEditorTasks on testwikidatawiki (duration: 00m 57s) [production]
21:22 <mdholloway> created new db tables for WikimediaEditorTasks in x1 [production]
21:00 <SMalyshev> depooled wdqs1006 to see if it'd catch up better [production]
20:19 <marxarelli> correction: group0 to 1.33.0-wmf.23 [production]
20:15 <dduvall@deploy1001> rebuilt and synchronized wikiversions files: group0 to 1.33.0-wmf.0 [production]
20:08 <ejegg> updated payments-wiki from f42910460b to 6661655e37 [production]
19:58 <dduvall@deploy1001> Finished scap: testwiki to php-1.33.0-wmf.23 and rebuild l10n cache (duration: 37m 59s) [production]
19:20 <dduvall@deploy1001> Started scap: testwiki to php-1.33.0-wmf.23 and rebuild l10n cache [production]
19:18 <marxarelli> scap clean failure due to T218783. train is rolling without cleanup [production]
19:17 <jynus> reloading db2095 mariadb instances to reload and check filters [production]
19:13 <jynus> reloading db2094 mariadb instances to reload and check filters [production]
19:07 <dduvall@deploy1001> clean aborted: Pruned MediaWiki: 1.33.0-wmf.17 (duration: 00m 10s) [production]
19:04 <jynus> reloading db1125 mariadb instances to reload and check filters [production]
18:49 <marxarelli> branch 1.33.0-wmf.23 was cut successfully (T206677) [production]
18:24 <jynus> reloading db1124 mariadb instances to reload and check filters [production]
18:21 <marxarelli> starting branch cut for 1.33.0-wmf.23 (T206677) [production]
18:09 <thcipriani> gerrit back on version 2.15.12, upgrade complete. [production]
18:05 <thcipriani> restarting gerrit on cobalt for update to 2.15.12 [production]
18:05 <gehel@cumin2001> END (FAIL) - Cookbook sre.elasticsearch.e6-upgrade (exit_code=99) [production]
18:05 <thcipriani@deploy1001> Finished deploy [gerrit/gerrit@d3d2134]: Gerrit to 2.15.12 on cobalt (duration: 00m 15s) [production]
18:04 <thcipriani@deploy1001> Started deploy [gerrit/gerrit@d3d2134]: Gerrit to 2.15.12 on cobalt [production]
18:03 <thcipriani@deploy1001> Finished deploy [gerrit/gerrit@d3d2134]: Gerrit to 2.15.12 on gerrit2001 only (duration: 00m 11s) [production]
18:03 <thcipriani@deploy1001> Started deploy [gerrit/gerrit@d3d2134]: Gerrit to 2.15.12 on gerrit2001 only [production]
18:01 <thcipriani> starting gerrit 2.15.12 upgrade [production]
17:45 <otto@deploy1001> scap-helm eventgate-analytics finished [production]
17:45 <otto@deploy1001> scap-helm eventgate-analytics cluster eqiad completed [production]
17:45 <otto@deploy1001> scap-helm eventgate-analytics upgrade production -f eventgate-analytics-eqiad-values.yaml --reset-values stable/eventgate-analytics [namespace: eventgate-analytics, clusters: eqiad] [production]
17:43 <otto@deploy1001> scap-helm eventgate-analytics finished [production]
17:43 <otto@deploy1001> scap-helm eventgate-analytics cluster codfw completed [production]
17:43 <otto@deploy1001> scap-helm eventgate-analytics upgrade production -f eventgate-analytics-codfw-values.yaml --reset-values stable/eventgate-analytics [namespace: eventgate-analytics, clusters: codfw] [production]
17:41 <otto@deploy1001> scap-helm eventgate-analytics finished [production]
17:41 <otto@deploy1001> scap-helm eventgate-analytics cluster staging completed [production]
17:41 <otto@deploy1001> scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml --reset-values stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] [production]
17:39 <otto@deploy1001> scap-helm eventgate-analytics finished [production]
17:39 <otto@deploy1001> scap-helm eventgate-analytics cluster staging completed [production]