|
2014-08-25
§
|
| 16:54 |
<ottomata> |
stopping puppet on cp3021. Testing an increase of http://kafka.queue.buffering.max.ms/ in order to avoid dropping messages during broker metadata change (e.g. leader elections) |
[production] |
| 16:48 |
<hashar> |
Jenkins pooled in a new slave [https://integration.wikimedia.org/ci/computer/wdjenkins-node1/ wdjenkins-node1] that will be used to run Wikidata jenkins jobs. Work in progress with addshore. It is not running jobs yet. |
[production] |
| 16:47 |
<godog> |
reboot ms-be1011, xfsaild errors in dmesg |
[production] |
| 16:25 |
<hashar> |
Jenkins: disconnecting and reconnecting Gearman plugin from https://integration.wikimedia.org/ci/configure |
[production] |
| 16:06 |
<andrewbogott> |
wikitech deployment finished. Note that the OpenStackManager submodule is off of the MediaWiki branch because… the whole submodule setup there is a bit broken on account of a git bug that uses absolute paths to manage submodules. |
[production] |
| 16:01 |
<andrewbogott> |
deploying tiny OpenStackManager upgrade on wikitech |
[production] |
| 15:58 |
<ottomata> |
enabled elasticsearch shard allocation row awareness (via rest api) |
[production] |
| 12:45 |
<hashar> |
hard stopped/restarted Zuul (workflow config error) |
[production] |
| 12:27 |
<hashar> |
restarting zuul |
[production] |
| 10:15 |
<mark> |
setup cross-confederation BGP sessions from AS65001 (eqiad) to AS65002 (codfw) |
[production] |
| 05:35 |
<hoo> |
Synchronized wmf-config/InitialiseSettings.php: {{gerrit|156076}} - Remove centralnotice-admin right assignments on 3 wikis - Basically a noop (duration: 00m 06s) |
[production] |
| 03:15 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Mon Aug 25 03:14:13 UTC 2014 (duration 14m 12s) |
[production] |
| 02:27 |
<LocalisationUpdate> |
completed (1.24wmf18) at 2014-08-25 02:25:58+00:00 |
[production] |
| 02:15 |
<LocalisationUpdate> |
completed (1.24wmf17) at 2014-08-25 02:14:14+00:00 |
[production] |
|
2014-08-23
§
|
| 11:33 |
<mark> |
Manually removed IPv6 addresses from fenari |
[production] |
| 11:23 |
<mark> |
Deactivated IPv6 router-advertisement on cr2-pmtpa |
[production] |
| 11:21 |
<mark> |
Manually removed IPv6 address from mchenry |
[production] |
| 10:36 |
<springle> |
Synchronized wmf-config/db-eqiad.php: depool db1004. pool db1053. (duration: 00m 07s) |
[production] |
| 03:07 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Sat Aug 23 03:06:13 UTC 2014 (duration 6m 12s) |
[production] |
| 02:23 |
<LocalisationUpdate> |
completed (1.24wmf18) at 2014-08-23 02:21:59+00:00 |
[production] |
| 02:18 |
<LocalisationUpdate> |
completed (1.24wmf17) at 2014-08-23 02:17:28+00:00 |
[production] |
| 01:32 |
<springle> |
Synchronized wmf-config/db-eqiad.php: repool db1056 (duration: 00m 06s) |
[production] |
| 00:29 |
<ori> |
disabled puppet on osmium again to debug a leak; please don't re-enable |
[production] |
|
2014-08-22
§
|
| 18:10 |
<ori> |
updated /a/common to {{Gerrit|I338d72a47}}: Do not define MEDIAWIKI before loading WebStart.php |
[production] |
| 17:43 |
<ottomata> |
moving sqstat udp2log filter from analytics1003 to analytics1026, reqstats might blip for a sec... |
[production] |
| 17:41 |
<ori> |
nuking /srv/deployment/rcstream on rcs1002 to verify trebuchet package provider reprovisions it |
[production] |
| 15:54 |
<springle> |
xtrabackup db1056 to db1053 |
[production] |
| 15:53 |
<springle> |
Synchronized wmf-config/db-eqiad.php: depool db1056 while cloning (duration: 00m 07s) |
[production] |
| 15:33 |
<^d> |
elastic1008: fixed /etc/hosts to point to actual IP instead of loopback |
[production] |
| 15:18 |
<springle> |
upgrade & restart db1053, fs check |
[production] |
| 15:08 |
<bd808> |
Still no apache2.log on fluorine or in logstash. Log seems to be available on fenari. |
[production] |
| 14:51 |
<springle> |
switched s1 sanitarium and labsdb replication to db1069:3311 mariadb 10 |
[production] |
| 14:39 |
<mark> |
Removed IPv6 subnets 2620:0:860:1::/64 (squid subnet) and 2620:0:860:3::/64 (sandbox subnet) from cr2-pmtpa configuration |
[production] |
| 04:11 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Fri Aug 22 04:10:47 UTC 2014 (duration 10m 46s) |
[production] |
| 03:19 |
<LocalisationUpdate> |
completed (1.24wmf18) at 2014-08-22 03:18:44+00:00 |
[production] |
| 02:33 |
<LocalisationUpdate> |
completed (1.24wmf17) at 2014-08-22 02:32:11+00:00 |
[production] |
| 00:03 |
<ori> |
Finished scap: SWAT: d3de89777, 7abfe0d5e7, 8ec9853c32b, 476e9e90bd01 (duration: 06m 29s) |
[production] |
|
2014-08-21
§
|
| 23:57 |
<ori> |
Started scap: SWAT: d3de89777, 7abfe0d5e7, 8ec9853c32b, 476e9e90bd01 |
[production] |
| 21:58 |
<ori> |
Synchronized php-1.24wmf17/resources/src/mediawiki/mediawiki.js: I8d27442d1: Workaround for bug introduced by Icf6ede09b (duration: 00m 03s) |
[production] |
| 21:57 |
<manybubbles> |
performing elasticsearch upgrade on elastic1015 |
[production] |
| 21:02 |
<ori> |
Synchronized php-1.24wmf17/resources/src/mediawiki/mediawiki.util.js: Touch resources/src/mediawiki/mediawiki.util.js (duration: 00m 06s) |
[production] |
| 20:44 |
<godog> |
rolling restart of swift-proxy on ms-fe1* |
[production] |
| 20:11 |
<godog> |
restarted swift-proxy on ms-fe1001 |
[production] |
| 19:55 |
<reedy> |
Synchronized wmf-config/: (no message) (duration: 00m 13s) |
[production] |
| 19:49 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.24wmf18 |
[production] |