651-700 of 10000 results (41ms)
2018-12-29 §
12:34 <bawolff@deploy1001> Synchronized private/PrivateSettings.php: T212667 - make spam mitigation global (duration: 00m 49s) [production]
11:52 <volans|off> manually recreated /dev/log symlink on kubernetes1001, restarting systemd-journald.socket didn't worked (this should fix cron-spam emails from the host every hour) [production]
11:43 <volans|off> restarted pdfrender on scb1004 [production]
09:21 <elukey> restart pdfrender on scb1004 [production]
2018-12-28 §
21:24 <Cam11598> restarted cvnbot18 [cvn]
21:24 <Cam11598> 1:23:04 PM <25B100+ CVNBot18> Last message was received on RCReader 31391.185322 seconds ago [cvn]
18:14 <bawolff@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 97446843a27 T212667 - Temp increase abusefilter emergency cutoff on mw.org to deal with spam attack (duration: 00m 46s) [production]
18:02 <bawolff@deploy1001> Synchronized private/PrivateSettings.php: T212667 - adjust account creation (duration: 00m 47s) [production]
15:28 <bawolff@deploy1001> Synchronized private/PrivateSettings.php: Attempt to adjust captcha settings for T212667 (duration: 00m 46s) [production]
05:41 <Cam11598> restarted cvnbot10 [cvn]
05:41 <Cam11598> 9:40:50 PM <25B100+ CVNBot10> Last message was received on RCReader 80417.36911 seconds ago [cvn]
05:37 <AlexZ> CVNBot3 restarted due to lag [cvn]
02:20 <tgr> rebuilding MW Discourse after applying https://github.com/tgr/discourse-chat-integration/commit/dffc22f9f71d77b41a3c794307841952eb5909b4 [discourse]
2018-12-27 §
22:50 <bd808> Deleted striker-deploy03 for T204563 [striker]
19:31 <mobrovac@deploy1001> Finished deploy [restbase/deploy@a67f38e]: Fix rate-limiter crash (with increased deploy delays), take #3 (duration: 25m 49s) [production]
19:05 <mobrovac@deploy1001> Started deploy [restbase/deploy@a67f38e]: Fix rate-limiter crash (with increased deploy delays), take #3 [production]
18:15 <mobrovac@deploy1001> Started deploy [restbase/deploy@a67f38e]: Fix rate-limiter crash (with increased deploy delays), take #2 [production]
18:15 <mobrovac@deploy1001> Finished deploy [restbase/deploy@a67f38e]: Fix rate-limiter crash (with increased deploy delays) - T212631 (duration: 11m 20s) [production]
18:04 <mobrovac@deploy1001> Started deploy [restbase/deploy@a67f38e]: Fix rate-limiter crash (with increased deploy delays) - T212631 [production]
18:04 <mobrovac@deploy1001> deploy aborted: Fix rate-limiter crash (with increased deploy delays) - T212631 (duration: 00m 09s) [production]
18:04 <mobrovac@deploy1001> Started deploy [restbase/deploy@a67f38e]: Fix rate-limiter crash (with increased deploy delays) - T212631 [production]
18:03 <mobrovac@deploy1001> deploy aborted: Fix rate-limiter crash - T212631 (duration: 13m 09s) [production]
17:50 <mobrovac@deploy1001> Started deploy [restbase/deploy@70c4752]: Fix rate-limiter crash - T212631 [production]
17:20 <mobrovac@deploy1001> Finished deploy [restbase/deploy@ae7a537]: Fix rate-limiter crash - T212631 - deploy only on canary restbase1007 (duration: 04m 24s) [production]
17:15 <mobrovac@deploy1001> Started deploy [restbase/deploy@ae7a537]: Fix rate-limiter crash - T212631 - deploy only on canary restbase1007 [production]
16:00 <cmjohnson1> powercycling frdb1001 for troubleshooting [production]
15:21 <bawolff> Updated interwiki cache https://biblio.wiki to https://wikilivres.org -> T212650 [production]
15:19 <bawolff@deploy1001> Synchronized wmf-config/interwiki.php: Updating interwiki cache (duration: 03m 34s) [production]
04:26 <bd808> Shutdown striker-deploy03 instance for T204563 [striker]
01:55 <Krinkle> Upgrade CVNBot18 to CVNBot 1.22.0-beta.1 (latest master, xbuild, cvn-app8) [cvn]
01:38 <Cam11598> restarted cvnbot18 had no response for over 30000 seconds [cvn]
00:24 <bd808> Promoted striker-deploy04 to active deploy server [striker]
2018-12-26 §
18:47 <Jayprakash12345> Setup cronjob to run script every 6 hours (T212517) [tools.indic-wsstats]
18:46 <Jayprakash12345> Setup cronjob to run script every 6 hours [tools.indic-wsstats]
18:46 <Jayprakash12345> tools-bastion-03:~/www/python/src$ git pull (Updating to 7c4c24e) (T212517) [tools.indic-wsstats]
16:37 <godog> clear logstash persistent queue /var/lib/logstash on logstash100[789] [production]
15:47 <godog> roll-restart logstash on logstash100[789] [production]
14:26 <filippo@puppetmaster1001> conftool action : set/pooled=no; selector: name=wtp1028.eqiad.wmnet [production]
14:18 <godog> powercycle wtp1028 - nothing on console [production]
09:13 <godog> swift eqiad-prod: more weight for ms-be10[44-50].eqiad.wmnet - T209618 [production]
07:20 <marostegui> Fix replication db1124:3318 [production]
2018-12-25 §
19:04 <Cam11598> CVNBot1 restarted [cvn]
19:04 <Cam11598> 11:04:09 AM <+CVNBot1> Last message was received on RCReader 24889.420019 seconds ago [cvn]
18:28 <Krinkle> CVNBot19 inactive over 5000 seconds. Restarted [cvn]
18:27 <Krinkle> CVNBot 6, 8, 9, 10 were inactive over 30,000 seconds. Restarted [cvn]
17:52 <Jayprakash12345> tools-bastion-03:~$ webservice --backend=kubernetes python start (T212517) [tools.indic-wsstats]
17:52 <Jayprakash12345> tools-bastion-03:~$ git clone https://github.com/Jayprakash-SE/indic-wsstats.git (T212517) [tools.indic-wsstats]
2018-12-24 §
21:40 <zhuyifei1999_> Deployed 2a51a54 on -web-01 T212598 [quarry]
05:38 <_joe_> powercycling lvs3001 [production]
01:43 <Cam11598> restarted all bots due to 6 minute lag reported by bsadaowski1 [cvn]