|
2017-03-31
§
|
| 14:56 |
<oblivian@puppetmaster1001> |
conftool action : set/ttl=10; selector: dnsdisc=restbase-async |
[production] |
| 14:55 |
<_joe_> |
reducing ttl on the restbase-async discovery record, then flipping eqiad to active |
[production] |
| 14:55 |
<volans> |
deploying the use of discovery URL to swift-proxy hosts in codfw T160178#3136906 |
[production] |
| 14:09 |
<_joe_> |
performing a rolling restart of changeprop after puppet runs on scb* |
[production] |
| 14:00 |
<oblivian@puppetmaster1001> |
conftool action : set/pooled=true; selector: dnsdisc=restbase-async,name=codfw |
[production] |
| 13:23 |
<elukey> |
restart hhvm on mw116[89] after https://gerrit.wikimedia.org/r/345829 |
[production] |
| 13:19 |
<gehel> |
rolling restart of maps-test cluster for kernel upgrade |
[production] |
| 13:09 |
<moritzm> |
rebooting bromine to Linux 4.9 |
[production] |
| 12:10 |
<moritzm> |
rebooting mwdebug* to Linux 4.9 |
[production] |
| 12:05 |
<moritzm> |
rebooting pybal-test* to Linux 4.9 |
[production] |
| 11:22 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1066 after maintenance (duration: 00m 49s) |
[production] |
| 10:47 |
<akosiaris> |
uploaded jessie-wikimedia kubernetes_1.4.6-4 on apt.wikimedia.org/jessie-wikimedia |
[production] |
| 09:59 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=mw2244.codfw.wmnet |
[production] |
| 09:56 |
<elukey> |
set pooled=yes mw210[56789], mw2260 and mw2213 (and cleaned up old /srv/mediawiki dirs that were causing rsync spam in scap pull) |
[production] |
| 09:52 |
<marostegui> |
Adding rev_timestamp index to revision page db1066 (s1) - T132416 |
[production] |
| 09:49 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1066 for maintenance (duration: 00m 44s) |
[production] |
| 09:47 |
<elukey> |
restart hhvm on mw1197 - hhvm dump debug in /tmp/hhvm.14540.bt. - threads stuck in Treadmill::getAgeOldestRequest (HHVM 3.12) |
[production] |
| 09:37 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1062 (duration: 00m 45s) |
[production] |
| 09:35 |
<godog> |
fix long-standing swift-account-server REPLICATE backtrace error on ms-be1022 - https://bugs.launchpad.net/swift/+bug/1424108 |
[production] |
| 09:21 |
<godog> |
delete stray nginx error log with debug logging on thumbor1002 |
[production] |
| 08:28 |
<moritzm> |
repooled mw1261 for more HHVM 3.18 debugging |
[production] |
| 07:29 |
<marostegui> |
Start pt-table-checksum on s5 wikidatawiki - T161294 |
[production] |
| 02:44 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Fri Mar 31 02:44:11 UTC 2017 (duration 5m 30s) |
[production] |
| 02:38 |
<l10nupdate@tin> |
scap sync-l10n completed (1.29.0-wmf.18) (duration: 15m 54s) |
[production] |
|
2017-03-30
§
|
| 23:58 |
<aude@tin> |
Synchronized php-1.29.0-wmf.18/extensions/Wikidata: Fixes for special pages (duration: 02m 15s) |
[production] |
| 23:07 |
<catrope@tin> |
Synchronized php-1.29.0-wmf.18/extensions/ORES/modules/: T161706 (duration: 00m 51s) |
[production] |
| 21:34 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group2 -> wmf.18 |
[production] |
| 21:16 |
<awight> |
reenabled ingenico orphan rectifier (jenkins) |
[production] |
| 21:08 |
<awight> |
disable ingenico orphan rectifier (jenkins) |
[production] |
| 21:07 |
<demon@tin> |
Synchronized php-1.29.0-wmf.18/extensions/Echo/includes/model/Event.php: fix logging class reference (duration: 00m 47s) |
[production] |
| 19:25 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group1 to wmf.18 |
[production] |
| 18:59 |
<MaxSem> |
Portals were not deployed: https://phabricator.wikimedia.org/T161832 |
[production] |
| 18:54 |
<maxsem@tin> |
Synchronized portals/: (no justification provided) (duration: 00m 48s) |
[production] |
| 18:45 |
<maxsem@tin> |
Synchronized portals: (no justification provided) (duration: 00m 44s) |
[production] |
| 18:45 |
<maxsem@tin> |
Synchronized portals/prod/wikipedia.org/assets: (no justification provided) (duration: 00m 43s) |
[production] |
| 18:38 |
<maxsem@tin> |
Synchronized portals: (no justification provided) (duration: 00m 44s) |
[production] |
| 18:29 |
<maxsem@tin> |
Synchronized portals: (no justification provided) (duration: 00m 45s) |
[production] |
| 18:28 |
<maxsem@tin> |
Synchronized portals/prod/wikipedia.org/assets: (no justification provided) (duration: 00m 44s) |
[production] |
| 18:24 |
<maxsem@tin> |
Synchronized portals: (no justification provided) (duration: 00m 44s) |
[production] |
| 18:24 |
<godog> |
swift eqiad-prod add ms-be1028 -> ms-be1039 - T160640 |
[production] |
| 18:23 |
<maxsem@tin> |
Synchronized portals/prod/wikipedia.org/assets: (no justification provided) (duration: 00m 44s) |
[production] |
| 18:18 |
<maxsem@tin> |
Synchronized portals: (no justification provided) (duration: 00m 44s) |
[production] |
| 18:17 |
<maxsem@tin> |
Synchronized portals/prod/wikipedia.org/assets: (no justification provided) (duration: 00m 45s) |
[production] |
| 17:32 |
<elukey> |
shutdown analytics1039 to apply new thermal paste - T132256 |
[production] |
| 16:17 |
<godog> |
upgrade thumbor to 0.1.37 on thumbor100[12] |
[production] |
| 16:03 |
<_joe_> |
restarting hhvm on mw1191, stuck in HPHP::Treadmill::getAgeOldestRequest |
[production] |
| 15:59 |
<twentyafterfour@tin> |
Synchronized php-1.29.0-wmf.17/includes/: sync I7c5c0a9b1af99ce2b5f4bdcc99710d8400ca8bcf refs T159319 (duration: 01m 41s) |
[production] |
| 14:50 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Remove db1057 entry from s1 shard - T160435 (duration: 00m 44s) |
[production] |
| 14:38 |
<godog> |
run stress test (w/ bonnie) on new swift hw - T160640 |
[production] |
| 14:33 |
<andrewbogott> |
upgrading nova-compute to 12.0.6 on all labvirts |
[production] |