|
2017-09-01
§
|
| 11:23 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1081 with low weight - T168661 (duration: 00m 44s) |
[production] |
| 10:35 |
<elukey> |
stop puppet on thorium and disable root rsyncs - T174756 |
[production] |
| 10:31 |
<marostegui> |
Upgrade MariaDB to 10.0.32 on db1081 - T168661 |
[production] |
| 10:29 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1081 - T168661 (duration: 00m 43s) |
[production] |
| 10:06 |
<elukey> |
killed root rsyncs on thorium, disabled puppet |
[analytics] |
| 09:10 |
<godog> |
bounce graphite-web on graphite1001, problematic query? using all CPU |
[production] |
| 09:04 |
<ema> |
lvs1007 upgrade to pybal 1.13.11 - one-packet-scheduling, instrumentation fixes. T104442, T103882 |
[production] |
| 08:59 |
<godog> |
depool restbase200[135] before reimage - T169939 |
[production] |
| 07:22 |
<elukey> |
restart apache2 and hue on thorium, Analytics sites down, investigating |
[production] |
| 07:06 |
<marostegui> |
Power reset db2044 as it is unresponsive - T174764 |
[production] |
| 03:54 |
<krinkle@tin> |
Synchronized wmf-config/InitialiseSettings.php: I4dfc33f66c3 - Enable jQuery 3 on nlwiki sister projects (duration: 00m 43s) |
[production] |
| 01:31 |
<ottomata> |
restarted hue (a few minutes ago) not totally sure why it died |
[analytics] |
| 00:57 |
<tstarling@tin> |
Synchronized php-1.30.0-wmf.16/includes/parser/Parser.php: (no justification provided) (duration: 00m 44s) |
[production] |
| 00:13 |
<RainbowSprinkles> |
gerrit: flushed all non-login caches, things might be sluggish for the next ~15mins or so |
[production] |
|
2017-08-31
§
|
| 23:32 |
<Krenair> |
fixed deployment-imagescaler0[12] puppet by installing a package and file manually, some puppetisation still needed - https://phabricator.wikimedia.org/T174746 |
[releng] |
| 23:14 |
<madhuvishy> |
Restarted webservice after fixing T174740 |
[tools.openstack-browser] |
| 23:04 |
<Krenair> |
that also did deployment-cache-(upload|text)04 |
[releng] |
| 22:50 |
<Krenair> |
fixed deployment-ms-be0[34] puppet by removing cherry-pick of https://gerrit.wikimedia.org/r/#/c/371582/1 - details in a comment there |
[releng] |
| 22:24 |
<Sagan> |
Oxygen: Updating Software |
[rcm] |
| 22:24 |
<Sagan> |
Neon: Restarting webservice |
[rcm] |
| 22:24 |
<Sagan> |
Tin: Updating Software |
[rcm] |
| 22:24 |
<Sagan> |
CAC: Running vagrant git-update |
[rcm] |
| 22:24 |
<Sagan> |
Xenon: Updating Software |
[rcm] |
| 22:24 |
<Sagan> |
Neon: Updating Software |
[rcm] |
| 21:12 |
<madhuvishy> |
Updated star.wmflabs.org cert, ran puppet and restarted nginx in novaproxy-01 and 02 (Copied the private key manually to /etc/ssl/private) T174611 |
[project-proxy] |
| 20:33 |
<madhuvishy> |
Updated certs and ran puppet, restarted nginx on tools-proxy-* and tools-static-* (T174611) |
[tools] |
| 20:25 |
<madhuvishy> |
Merging new cert https://gerrit.wikimedia.org/r/#/c/374873/ (T174611) |
[tools] |
| 20:24 |
<madhuvishy> |
Disabling puppet on tools-proxy-* and tools-static-* for star.wmflabs.org SSL cert update (T174611) |
[tools] |
| 20:23 |
<madhuvishy> |
Disabling puppet on tools-proxy-* and tools-static-* for star.wmflabs.org SSL cert update |
[tools] |
| 20:03 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: (no justification provided) |
[production] |
| 19:27 |
<mattflaschen@tin> |
Finished scap: Watchlist filters: Convert edit watchlist button to new UX and fix server-side tag filtering. T172030 (duration: 21m 23s) |
[production] |
| 19:17 |
<cmjohnson1> |
powercycled mw1294 |
[production] |
| 19:05 |
<mattflaschen@tin> |
Started scap: Watchlist filters: Convert edit watchlist button to new UX and fix server-side tag filtering. T172030 |
[production] |
| 18:53 |
<mattflaschen@tin> |
Synchronized docroot/noc/conf/interwiki-labs.php.txt: Interwiki links on Beta Cluster: T69931 (duration: 00m 47s) |
[production] |
| 18:51 |
<mattflaschen@tin> |
Synchronized wmf-config: Interwiki links on Beta Cluster: T69931 (duration: 00m 49s) |
[production] |
| 17:50 |
<demon@tin> |
Synchronized php-1.30.0-wmf.16/skins/MinervaNeue/resources/skins.minerva.icons.images.scripts/watch.svg: sorta impromptu swat thing (duration: 00m 47s) |
[production] |
| 16:35 |
<ejegg> |
re-enabled donation queue consumer and thank you mail sender |
[production] |
| 15:49 |
<thcipriani> |
reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/374970/ |
[releng] |
| 15:25 |
<ejegg> |
updated civicrm from f00611d9aa3ae4a1ffbff70dcca90c7ae07b6c68 to 820bd98c7c23a851c38a75111c8c1532c781e198 |
[production] |
| 15:20 |
<ejegg> |
updated SmashPig from 8f4388d407f313c1083cf96ed106efcc5d5a29f7 to 8eb98c10ff1129ea65dd81559656001c819cc12f |
[production] |
| 15:19 |
<ejegg> |
turned off donation queue consumer and thank you job |
[production] |
| 15:18 |
<elukey> |
restart zookeeper on conf200[2,3] for jvm security updates |
[production] |
| 14:52 |
<zeljkof> |
Reloading Zuul to deploy dd504839b3a48f431199785421751a5e3e38af57 |
[releng] |
| 13:52 |
<Amir1> |
5cc380f is going to staging |
[wikilabels] |
| 13:36 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Restore db1084 original weight - T168661 (duration: 00m 47s) |
[production] |
| 13:10 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase db1084 weight - T168661 (duration: 00m 47s) |
[production] |
| 13:07 |
<elukey> |
restart zookeeper on conf2001 for security updates (canary node) |
[production] |
| 13:00 |
<moritzm> |
installing augeas security updates |
[production] |
| 12:34 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1084 with low weight - T168661 (duration: 00m 47s) |
[production] |
| 12:14 |
<marostegui> |
Upgrade MariaDB on db1084 - T168661 |
[production] |