251-300 of 10000 results (37ms)
2019-08-26 §
13:06 <mobrovac@deploy1001> deploy aborted: Bring the dev cluster up to date and expose RB on both 7231 and 7233 in it - T223953 (duration: 00m 04s) [production]
13:06 <mobrovac@deploy1001> Started deploy [restbase/deploy@38c313d]: Bring the dev cluster up to date and expose RB on both 7231 and 7233 in it - T223953 [production]
12:48 <marostegui> Restart MySQL on db2114 to pick up binlog format change [production]
12:48 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Clarify db2114 status (duration: 00m 45s) [production]
11:57 <mobrovac@deploy1001> Finished deploy [cpjobqueue/deploy@e742ecf]: Increase the concurrency of cirusSearchCheckerJobs to 20 - T231194 (duration: 01m 31s) [production]
11:55 <mobrovac@deploy1001> Started deploy [cpjobqueue/deploy@e742ecf]: Increase the concurrency of cirusSearchCheckerJobs to 20 - T231194 [production]
11:36 <Amir1> EU SWAT is done [production]
11:34 <ladsgroup@deploy1001> Synchronized php-1.34.0-wmf.19/extensions/UniversalLanguageSelector: SWAT: [[gerrit:532341|Revert "Return target of redirect languages in mw.uls.getFrequentLanguageList" (T217770 T121747)]] (duration: 00m 46s) [production]
11:30 <joal> Remove tracking failure for http://Wikipedia/screen/Explore in matomo [analytics]
11:10 <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:527087|Switch property terms migration to WRITE_NEW on client wikis (T225053)]] (duration: 00m 46s) [production]
10:47 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:528092| Bumping portals to master (T128546)]] (duration: 00m 46s) [production]
10:47 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:528092| Bumping portals to master (T128546)]] (duration: 00m 46s) [production]
10:26 <vgutierrez> uploaded trafficserver-8.0.5-1wm2 to apt.wikimedia.org (stretch) - T221594 [production]
09:54 <_joe_> codfw/appserver/*/mw2231.codfw.wmnet: pooled changed yes => inactive T231192 [production]
09:43 <Urbanecm> Run scap pull on mwdebug1001, test ended [production]
09:38 <Urbanecm> Enable partial blocks on test2wiki and mwdebug1001 to test something [production]
08:46 <_joe_> hard powercycle of mw2231, down with a blank console [production]
06:51 <ema> cp-upload: rolling ats-backend-restart to enable compress plugin [production]
05:25 <marostegui> Upload new mariadb 10.3 packages to repo [production]
05:09 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool pc2009 for optimize T210725 (duration: 02m 53s) [production]
05:08 <marostegui> Optimize tables on pc2009 - T210725 [production]
2019-08-25 §
21:27 <Reedy> centralauth.globalblocks `ALTER TABLE /*_*/globalblocks MODIFY gb_reason varbinary(767) NOT NULL; [releng]
16:43 <Urbanecm> Revert 'Revert 45831e6 locally on beta cluster to test possible root cause of T231162' [releng]
16:33 <Urbanecm> Revert 45831e6 locally on beta cluster to test possible root cause of T231162 [releng]
16:33 <Urbanecm> Revert 45831e6 locally on beta cluster to test possible root cause of T231162 [deployment-prep]
13:46 <volans> uploaded spicerack_0.0.27-1_amd64.deb to apt.wikimedia.org stretch-wikimedia [production]
02:22 <cdanis> clear downtimes on cr2-eqiad/cr2-codfw, link supposedly stable now [production]
00:35 <herron> set icinga downtimes on flapping cr2-eqiad and cr2-codfw alerts until monday [production]
2019-08-24 §
16:58 <Krinkle> cvn-app9 is consistently running out of memory while cvn-app8 is fairly flat. Maybe one of the bots is leaking memory or too many bots on one server? Which one? [cvn]
16:48 <Krinkle> Renamed SWBot3 to CVNBot15 (#cvn-simplewikis) [cvn]
15:27 <Urbanecm> Run mwscript importImages.php --wiki=commonswiki --comment-ext=txt --user=Shangkuanlc /home/urbanecm/T231129 (T231129) [production]
10:28 <wm-bot> <maurelio> Restarted SULWatcher [tools.stewardbots]
2019-08-23 §
23:34 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
23:34 <robh@cumin1001> START - Cookbook sre.hosts.decommission [production]
23:03 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
23:03 <robh@cumin1001> START - Cookbook sre.hosts.decommission [production]
22:05 <eileen> process-control config revision is 8c900d909f [production]
21:48 <XioNoX> increase ospf cost of zayo codfw-eqiad link to 1320 (was 320) to make it secondary [production]
20:52 <wikibugs> Updated channels.yaml to: ff7f5fed938e91a41c91200c105f00dc6b3142f1 Reading-Infrastructure-Team Phab boards have new names [tools.wikibugs]
19:17 <bd808> Update to f742092 Update text on error pages (T231090) [tools.admin]
19:11 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
19:11 <robh@cumin1001> START - Cookbook sre.hosts.decommission [production]
18:14 <James_F> Dropped 2FA for User:DBrant (WMF), per request. [production]
17:40 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
17:39 <robh@cumin1001> START - Cookbook sre.hosts.decommission [production]
12:24 <joal> Rerunning refine for eventlogging-analytics for 2019-08-23T03:00 [analytics]
12:24 <_joe_> pooling mw1270 temporarily, debugging performance issues [production]
12:15 <_joe_> depooling mw1270 temporarily, performance issues [production]
11:13 <marostegui> Upgrade db1114 from 10.3.16 to 10.3.17 [production]
11:12 <Urbanecm> deployment-prep: run scap sync to ensure everything is deployed (T231058) [releng]