5051-5100 of 10000 results (58ms)
2018-10-10 §
10:54 <marostegui> Set a replication filter on db1075 (s3 eqiad) to ignore enwikivoyage, cebwiki, shwiki, srwiki & mgwiktionary - T184805 [production]
10:49 <marostegui@deploy1001> Synchronized dblists/s5.dblist: Update s5.dblist to reflect the wikis moved from s3 - T184805 (duration: 00m 56s) [production]
10:48 <marostegui@deploy1001> Synchronized dblists/s3.dblist: Update s3.dblist to reflect the wikis moved to s5 - T184805 (duration: 00m 58s) [production]
09:12 <ema> Traffic: move restbase back to eqiad T203777 [production]
09:07 <ema> Traffic: set services active/active T203777 [production]
09:00 <ema> Traffic: route esams caches back to eqiad T203777 [production]
08:27 <moritzm> installing fuse security updates [production]
08:07 <ariel@deploy1001> Finished deploy [dumps/dumps@0714a93]: fix adds/changes dumps generation when prev run is missing (duration: 00m 06s) [production]
08:07 <ariel@deploy1001> Started deploy [dumps/dumps@0714a93]: fix adds/changes dumps generation when prev run is missing [production]
08:01 <moritzm> rolling out debdeploy 0.0.99.6 [production]
07:51 <elukey> cleaned up some log files from eventlog1002 [production]
02:55 <ejegg> updated payments-wiki from 1472604b6e to 7fb1aae963 [production]
00:19 <krinkle@deploy1001> Synchronized php-1.32.0-wmf.24/includes/utils/UIDGenerator.php: T94522 - I2a0c51bea58 (duration: 00m 56s) [production]
00:15 <krinkle@deploy1001> sync-file aborted: T205567 - I75f1eb6dc2cb (duration: 00m 01s) [production]
00:14 <krinkle@deploy1001> Synchronized php-1.32.0-wmf.24/tests/phpunit/includes/utils/: T94522 - I2a0c51bea58 (duration: 01m 02s) [production]
2018-10-09 §
22:58 <SMalyshev> repooled wdqs2003 [production]
22:26 <shdubsh> repairing /dev/sdl1 on ms-be2040 - T199198 [production]
21:52 <bblack> cp1085: varnish backend restart for mbox lag [production]
21:50 <mutante> releases1001 - restarted jenkins (it went from 200 -> 503 -> 403) curl localhost:8080 works again after restart, icinga check still getting 403 now [production]
21:41 <ejegg|food> updated fundraising CiviCRM from 7a0d14015e to 1165e7ed79 [production]
20:08 <mutante> repair /dev/sdg1 on ms-be2041 - T199198 [production]
19:37 <XioNoX> disable igmp-snooping on asw2-c-eqiad - T201039 [production]
19:25 <XioNoX> disable igmp-snooping on asw2-b-eqiad - T201039 [production]
19:20 <XioNoX> bounce igmp-snooping on asw2-b-eqiad [production]
18:24 <ottomata> adding Accept header to all varnishkafka generated webrequest logs [production]
17:21 <SMalyshev> depooled wdq23 again, sigh [production]
13:54 <moritzm> rebooting prometheus1004 for kernel security update [production]
13:41 <moritzm> rebooting prometheus1003 for kernel security update [production]
13:28 <moritzm> rebooting prometheus2004 for kernel security update [production]
13:13 <moritzm> rebooting prometheus2003 for kernel security update [production]
12:54 <gehel> silencing wdqs-public lag alerts (service still functional, and SLO unclear) - T199228 [production]
12:45 <moritzm> installing imagemagick security updates [production]
11:47 <END> (ERROR) - Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (exit_code=2) (volans@neodymium) [production]
11:46 <START> - Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (volans@neodymium) [production]
11:45 <akosiaris> dry-run services switchover from codfw to eqiad in preparation for Thursday [production]
11:37 <END> (ERROR) - Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (exit_code=2) (volans@neodymium) [production]
11:37 <START> - Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (volans@neodymium) [production]
11:14 <volans> live-test of the inverted switchdc (eqiad->codfw) completed, all good - T203777 [production]
11:14 <END> (PASS) - Cookbook sre.switchdc.mediawiki.08-update-tendril (exit_code=0) (volans@neodymium) [production]
11:13 <START> - Cookbook sre.switchdc.mediawiki.08-update-tendril (volans@neodymium) [production]
11:12 <END> (PASS) - Cookbook sre.switchdc.mediawiki.08-start-maintenance (exit_code=0) (volans@neodymium) [production]
11:11 <START> - Cookbook sre.switchdc.mediawiki.08-start-maintenance (volans@neodymium) [production]
11:11 <END> (PASS) - Cookbook sre.switchdc.mediawiki.08-restore-ttl (exit_code=0) (volans@neodymium) [production]
11:11 <START> - Cookbook sre.switchdc.mediawiki.08-restore-ttl (volans@neodymium) [production]
11:11 <END> (PASS) - Cookbook sre.switchdc.mediawiki.07-set-readwrite (exit_code=0) (volans@neodymium) [production]
11:11 <[DRY-RUN]> MediaWiki read-only period ends at: 2018-10-09 11:11:05.042622 (volans@neodymium) [production]
11:11 <START> - Cookbook sre.switchdc.mediawiki.07-set-readwrite (volans@neodymium) [production]
11:08 <END> (PASS) - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite (exit_code=0) (volans@neodymium) [production]
11:08 <START> - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite (volans@neodymium) [production]
11:07 <END> (PASS) - Cookbook sre.switchdc.mediawiki.05-invert-redis-sessions (exit_code=0) (volans@neodymium) [production]