1251-1300 of 10000 results (28ms)
2020-11-18 §
06:00 <marostegui@cumin1001> dbctl commit (dc=all): 'es1018 (re)pooling @ 25%: Slowly pool es1018 after cloning es1032 T261717', diff saved to https://phabricator.wikimedia.org/P13307 and previous config saved to /var/cache/conftool/dbconfig/20201118-060045-root.json [production]
05:47 <marostegui> Run check table on enwiki on db1124:3311 T267090 [production]
05:45 <marostegui@cumin1001> dbctl commit (dc=all): 'es1018 (re)pooling @ 10%: Slowly pool es1018 after cloning es1032 T261717', diff saved to https://phabricator.wikimedia.org/P13306 and previous config saved to /var/cache/conftool/dbconfig/20201118-054542-root.json [production]
00:53 <tgr_> also deployed [[gerrit:641294|Suggested Edits: Guard against task type not existing (T268012)]] [production]
00:52 <tgr@deploy1001> Synchronized php-1.36.0-wmf.18/extensions/GrowthExperiments/includes/HomepageModules/SuggestedEdits.php: Backport: [[gerrit:641295|Suggested edits: Guard against empty topic data (T268015)]] (duration: 01m 07s) [production]
00:27 <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:641250|Enable watchlist expiry feature on Wikidata & Commons (T266874)]] (duration: 01m 03s) [production]
2020-11-17 §
22:54 <mforns@deploy1001> Finished deploy [analytics/refinery@f19d20c] (thin): Regular analytics weekly train THIN [analytics/refinery@f19d20c21ada05df230d00c6e0022a7d5c356c13] (duration: 00m 07s) [production]
22:54 <mforns@deploy1001> Started deploy [analytics/refinery@f19d20c] (thin): Regular analytics weekly train THIN [analytics/refinery@f19d20c21ada05df230d00c6e0022a7d5c356c13] [production]
22:53 <mforns@deploy1001> Finished deploy [analytics/refinery@f19d20c]: Regular analytics weekly train [analytics/refinery@f19d20c21ada05df230d00c6e0022a7d5c356c13] (duration: 12m 51s) [production]
22:45 <clarakosi@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'mathoid' for release 'production' . [production]
22:40 <mforns@deploy1001> Started deploy [analytics/refinery@f19d20c]: Regular analytics weekly train [analytics/refinery@f19d20c21ada05df230d00c6e0022a7d5c356c13] [production]
22:39 <clarakosi@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'mathoid' for release 'production' . [production]
22:29 <clarakosi@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'mathoid' for release 'staging' . [production]
22:10 <mutante> otrs1001 - systemctl start otrs-cache-cleanup [production]
22:08 <ppchelko@deploy1001> Finished deploy [restbase/deploy@8363aeb]: update to service-runner 2.8.0, everywhere (duration: 11m 07s) [production]
22:07 <mutante> otrs1001 - removing otrs-cache-cleanup cron from otrs's crontab - adding same command as systemd timer. gerrit:637038 T265138 [production]
21:57 <ppchelko@deploy1001> Started deploy [restbase/deploy@8363aeb]: update to service-runner 2.8.0, everywhere [production]
21:32 <ppchelko@deploy1001> Finished deploy [restbase/deploy@8363aeb]: update to service-runner 2.8.0, codfw (duration: 07m 11s) [production]
21:24 <ppchelko@deploy1001> Started deploy [restbase/deploy@8363aeb]: update to service-runner 2.8.0, codfw [production]
20:56 <dancy@deploy1001> rebuilt and synchronized wikiversions files: group0 wikis to 1.36.0-wmf.18 [production]
20:42 <Urbanecm> End of mwscript extensions/AbuseFilter/maintenance/updateVarDumps.php --wiki=$wiki --print-orphaned-records-to=/tmp/urbanecm/$wiki-orphaned.log --progress-markers > $wiki.log in a tmux at mwmaint1002 (wiki=itwiki; T246539) [production]
20:31 <dancy@deploy1001> Finished scap: testwikis wikis to 1.36.0-wmf.18 (duration: 39m 37s) [production]
19:58 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
19:56 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
19:52 <dancy@deploy1001> Started scap: testwikis wikis to 1.36.0-wmf.18 [production]
19:50 <ppchelko@deploy1001> Finished deploy [restbase/deploy@8363aeb]: update to service-runner 2.8.0, canary on 2010 (duration: 02m 03s) [production]
19:48 <ppchelko@deploy1001> Started deploy [restbase/deploy@8363aeb]: update to service-runner 2.8.0, canary on 2010 [production]
19:46 <dancy@deploy1001> Pruned MediaWiki: 1.36.0-wmf.11 (duration: 13m 05s) [production]
19:24 <elukey@cumin1001> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) [production]
19:21 <elukey@cumin1001> START - Cookbook sre.hosts.decommission [production]
19:18 <elukey@cumin1001> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) [production]
19:12 <otto@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: wgEventStreamsDefaultSettings in beta should only set eqiad as topic prefix - T253069 (duration: 02m 26s) [production]
19:12 <elukey@cumin1001> START - Cookbook sre.hosts.decommission [production]
19:09 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
19:07 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
18:38 <ejegg> updated standalone SmashPig deployment from 09f29c1da5 to 63dffcb11f [production]
18:36 <ejegg> updated fundraising python tools from 68e054c9ad to 41cab089da [production]
18:09 <jynus> stopping db1139 for hw maintenance T261405 [production]
17:59 <dpifke@deploy1001> Finished deploy [performance/navtiming@8eaf7db]: (no justification provided) (duration: 00m 05s) [production]
17:58 <dpifke@deploy1001> Started deploy [performance/navtiming@8eaf7db]: (no justification provided) [production]
17:37 <dpifke@deploy1001> Finished deploy [performance/coal@43b91df]: (no justification provided) (duration: 00m 06s) [production]
17:37 <dpifke@deploy1001> Started deploy [performance/coal@43b91df]: (no justification provided) [production]
17:34 <dpifke@deploy1001> Finished deploy [statsv/statsv@249d073]: (no justification provided) (duration: 00m 05s) [production]
17:34 <dpifke@deploy1001> Started deploy [statsv/statsv@249d073]: (no justification provided) [production]
17:27 <dpifke@deploy1001> Finished deploy [statsv/statsv@873ea90]: (no justification provided) (duration: 00m 05s) [production]
17:27 <dpifke@deploy1001> Started deploy [statsv/statsv@873ea90]: (no justification provided) [production]
17:19 <dzahn@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
17:16 <dpifke@deploy1001> Finished deploy [performance/arc-lamp@55d4d41]: (no justification provided) (duration: 00m 04s) [production]
17:16 <dpifke@deploy1001> Started deploy [performance/arc-lamp@55d4d41]: (no justification provided) [production]
17:15 <dpifke@deploy1001> Finished deploy [performance/arc-lamp@55fccc6]: (no justification provided) (duration: 00m 04s) [production]