1351-1400 of 10000 results (56ms)
2017-11-27 §
18:11 <gehel> deploying blazegraph + GUI updates [production]
18:10 <gehel@tin> Started deploy [wdqs/wdqs@7ad20f3]: (no justification provided) [production]
18:08 <hoo> Killed truthy nt dumpers stuck at 100% CPU (from last week) on snapshot1007 (T181385) [production]
18:06 <akosiaris> poweroff ganeti1005, ganeti1006 T181121 [production]
18:05 <bd808> Sending Toolforge survey 1 week reminder emails from silver for T177126 [production]
16:07 <bblack> lvs400x - puppet disabled for https://gerrit.wikimedia.org/r/#/c/393610 [production]
15:52 <herron> beginning canary cutover/deployment of codfw prometheus servers to codfw puppet 4 puppetmasters [production]
15:19 <chasemp> disable puppet across cloud things for cleanup [production]
15:11 <herron> beginning canary cutover/deployment of codfw elasticsearch servers to codfw puppet 4 puppetmasters [production]
14:22 <zeljkof> EU SWAT finished [production]
14:19 <zfilipin@tin> Synchronized wmf-config/throttle.php: SWAT: [[gerrit:393442|IP cap lift for Semaine contributive 2017-2018 (T181360)]] (duration: 00m 45s) [production]
14:10 <addshore@tin> Synchronized php-1.31.0-wmf.8/extensions/AdvancedSearch: SWAT AdvancedSearch T181175 T181222, adjust links in beta section & fix usability of mobile search (duration: 00m 46s) [production]
13:31 <moritzm> installing nspr security updates on trusty [production]
13:22 <elukey> remove eventlogging replication support (log database) from dbstore1002 - T156844 [production]
13:17 <moritzm> updating nginx on francium [production]
13:01 <Pchelolo> stop cpjobqueue in eqiad for backlog accumulation [production]
12:44 <Pchelolo> started parsoid linter script to generate load on cpjobqueue (python3 parsoid_reparse.py http://parsoid.discovery.wmnet:8000 --sitematrix --linter-only --skip-closed https://commons.wikimedia.org/w/api.php) [production]
12:44 <addshore@tin> Synchronized wmf-config/InitialiseSettings-labs.php: [[gerrit:393587]] BETA wmgMonologChannels FileImporter => debug (duration: 01m 01s) [production]
12:43 <jmm@puppetmaster1001> conftool action : set/pooled=inactive; selector: mw1276.eqiad.wmnet [production]
12:35 <moritzm> powercycling mw1276 [production]
12:21 <addshore@tin> Synchronized wmf-config/CommonSettings-labs.php: [[gerrit:393582]] LABS ONLY Actually call wfLoadExtension for FileExporter & Importer on beta BETA (T181383) (duration: 00m 44s) [production]
12:14 <marostegui> Stop replication on db1109 to test table rename for s5/s8 failover [production]
12:09 <ppchelko@tin> Finished deploy [cpjobqueue/deploy@47d27dc]: Enable keep-alive T181007 (duration: 01m 03s) [production]
12:08 <ppchelko@tin> Started deploy [cpjobqueue/deploy@47d27dc]: Enable keep-alive T181007 [production]
11:47 <addshore@tin> Synchronized wmf-config/CommonSettings-labs.php: [[gerrit:393577]] LABS ONLY Enable FileImporter & FileExporter on BETA PT2/2 (T181383) (duration: 00m 45s) [production]
11:46 <addshore@tin> Synchronized wmf-config/InitialiseSettings-labs.php: [[gerrit:393577]] LABS ONLY Enable FileImporter & FileExporter on BETA PT1/2 (T181383) (duration: 00m 45s) [production]
11:44 <addshore@tin> Synchronized wmf-config/extension-list-labs: [[gerrit:393576]] BETA ONLY Add FileExporter & FileImporter to extension-list-labs (duration: 00m 45s) [production]
11:27 <hashar> contint1001 enable Icinga "Disks space" notification again. It is no more complaing about Docker partitions | ping mutante | T178454 [production]
11:17 <godog> bootstrap restbase1007-a - T179422 [production]
11:05 <jdrewniak@tin> Synchronized portals: Wikimedia Portals Update: [[gerrit:393559|Bumping portals to master (T128546)]] (duration: 00m 45s) [production]
11:05 <jdrewniak@tin> Synchronized portals/prod/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:393559|Bumping portals to master (T128546)]] (duration: 00m 46s) [production]
10:53 <moritzm> installing postgresql-common security updates [production]
10:08 <ppchelko@tin> Finished deploy [cpjobqueue/deploy@e35aa05]: Revert using keep-alive (duration: 00m 22s) [production]
10:08 <ppchelko@tin> Started deploy [cpjobqueue/deploy@e35aa05]: Revert using keep-alive [production]
09:42 <ppchelko@tin> Finished deploy [cpjobqueue/deploy@b570d4e]: Make http agent use keep-alive (duration: 00m 48s) [production]
09:41 <ppchelko@tin> Started deploy [cpjobqueue/deploy@b570d4e]: Make http agent use keep-alive [production]
09:14 <godog> reimage restbase1007 - T179422 [production]
08:55 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Fully pool db1097:3314 - T178359 (duration: 00m 43s) [production]
08:40 <moritzm> installing openjdk security updates on hadoop, druid and kafka clusters [production]
08:27 <marostegui> Deploy schema change on dbstore1002 and dbstore1001 - T174569 [production]
08:22 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1097:3314 weight - T178359 (duration: 00m 45s) [production]
07:20 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1097:3314 weight - T178359 (duration: 00m 45s) [production]
07:10 <marostegui> Stop MySQL on db1021 as it will be decommissioned - T181378 [production]
06:53 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Remove db1021 from the config as it will be decommissioned - T181378 (duration: 00m 44s) [production]
06:52 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove db1021 from the config as it will be decommissioned - T181378 (duration: 00m 45s) [production]
06:27 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Pool db1097:3314 with low weight - T178359 (duration: 00m 46s) [production]
02:22 <l10nupdate@tin> scap sync-l10n completed (1.31.0-wmf.8) (duration: 05m 44s) [production]
2017-11-25 §
19:31 <marostegui> Set 32:3 disk to offline on db1051 [production]
16:09 <kartik@tin> Finished deploy [cxserver/deploy@11aecc9]: Update cxserver to 0c242c0, Pin service-runner to 2.4.2 (duration: 03m 29s) [production]
16:05 <kartik@tin> Started deploy [cxserver/deploy@11aecc9]: Update cxserver to 0c242c0, Pin service-runner to 2.4.2 [production]