2151-2200 of 10000 results (50ms)
2017-12-05 §
17:27 <jmm@puppetmaster1001> conftool action : set/pooled=yes; selector: mw2118.codfw.wmnet [production]
15:56 <awight> beginning stress test on ores* (non-production) [production]
15:54 <awight@tin> Finished deploy [ores/deploy@6baed71]: (non-production) Test ORES deployment to ores100[1-2] (duration: 01m 01s) [production]
15:53 <awight@tin> Started deploy [ores/deploy@6baed71]: (non-production) Test ORES deployment to ores100[1-2] [production]
15:15 <ottomata> restarrting kafka-jumbo brokers, applying SSL (downtime scheduled) [production]
15:03 <urandom> bootstrapping cassandra, restbase1014-c - T179422 [production]
14:54 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Fully repool db1076 (duration: 00m 43s) [production]
14:41 <zeljkof> EU SWAT finished [production]
14:40 <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:395540|Revert "Add category collation for sewiki" (T181503)]] (duration: 00m 44s) [production]
14:15 <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:395009|Enable RemexHTML on itwiki and dewiki (T181188 T181190)]] (duration: 00m 43s) [production]
13:39 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1076 weight (duration: 00m 44s) [production]
13:14 <moritzm> reimaging mw2118/mw2119 (video scalers) to stretch [production]
12:21 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Fully repool db1034 (duration: 00m 43s) [production]
12:13 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1076 with low weight (duration: 00m 44s) [production]
11:58 <marostegui> Upgrade MariaDB and kernel on db1076 [production]
11:53 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1034 - T178359! (duration: 00m 43s) [production]
10:53 <moritzm> rebooting meitnerium/archiva.wikimedia.org for update to 4.9.51 [production]
10:46 <moritzm> rebooting einsteinium (icinga.wikimedia.org) for update to 4.9,51 [production]
10:45 <elukey> reboot druid1003 for kernel+jvm updates - T179943 [production]
10:33 <addshore@tin> Synchronized wmf-config/extension-list: [[gerrit:394283|extension-list extension.json entrypoint for ArticlePlaceholder]] (duration: 00m 43s) [production]
10:33 <moritzm> rebooting tegmen for update to 4.9,51 [production]
10:07 <jynus> stopping dbstore1002 (s5) and dbstore2001 (s5) for maintenance [production]
10:05 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1034 - T178359! (duration: 00m 43s) [production]
10:00 <ema> cp4021: restart varnish-be due to mbox lag/fetch failures [production]
09:49 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1034 with low weight - T178359! (duration: 00m 43s) [production]
09:42 <elukey> reboot analytics100[12] for kernel+jvm updates (Hadoop Master nodes) - T179943 [production]
09:39 <godog> bootstrap restbase1014-b - T179422 [production]
09:20 <marostegui> Optimize s7 on db1098 - T178359 [production]
09:03 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1076 - T174569 (duration: 00m 43s) [production]
09:01 <marostegui> Deploy schema change on db1076 (s2) - T174569 [production]
08:57 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1090 - T174569 (duration: 00m 44s) [production]
08:54 <moritzm> enabling test production traffic for mw1259 (stretch-based video scaler) [production]
08:24 <jmm@puppetmaster1001> conftool action : set/pooled=yes; selector: mw1259.eqiad.wmnet [production]
08:08 <moritzm> installing python updates on trusty [production]
07:27 <kartik@tin> Finished deploy [cxserver/deploy@4b74f03]: Update cxserver to 1693bcf (duration: 03m 22s) [production]
07:23 <kartik@tin> Started deploy [cxserver/deploy@4b74f03]: Update cxserver to 1693bcf [production]
06:58 <marostegui> Stop MySQL on db1034 to clone db1098:3317 - T178359 [production]
06:55 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1034 - T178359! (duration: 00m 43s) [production]
06:48 <marostegui> Fix dbstore1002 replication [production]
06:20 <marostegui> Deploy schema change on db1090 (s2) - T174569 [production]
06:20 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1090 - T174569 (duration: 00m 43s) [production]
06:15 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1103:3312 - T174569 (duration: 00m 44s) [production]
02:29 <l10nupdate@tin> scap sync-l10n completed (1.31.0-wmf.10) (duration: 06m 36s) [production]
00:29 <mutante> bast4001 - removing ganglia aggregators, package, config... [production]
2017-12-04 §
23:44 <mutante> bast3002 - killall -u ganglia to kill all aggregator procs, apt-get remove --purge ganglia-monitor, rm -rf /etc/ganglia, rm -rf /usr/lib/ganglia, apt-get autoremove [production]
23:28 <anomie@tin> Synchronized wmf-config/InitialiseSettings.php: Revert wgCommentTableSchemaMigrationStage change, breaks too much stuff (duration: 00m 44s) [production]
22:08 <brion> brion running requeueTranscodes.php on terbium to batch-run .mp3 output on Commons (T181749) [production]
21:27 <ppchelko@tin> Finished deploy [cpjobqueue/deploy@c8bea2e]: (no justification provided) (duration: 00m 42s) [production]
21:26 <ppchelko@tin> Started deploy [cpjobqueue/deploy@c8bea2e]: (no justification provided) [production]
21:26 <awight@tin> Finished deploy [ores/deploy@6baed71]: Update ORES to 6baed71 (duration: 12m 54s) [production]