|
2016-10-07
§
|
| 23:16 |
<Krenair> |
test morebots |
[production] |
| 23:15 |
<thcipriani@tin> |
Synchronized w/robots.php: Revert change to robots.php (duration: 00m 49s) |
[production] |
| 22:22 |
<mutante> |
etcd servers have puppet issue with Etcd_user[root] |
[production] |
| 22:04 |
<ejegg> |
updated payments-wiki from 1fbe171bca30d1fb7a4b7d937a740d87f7c9c8e3 to b4ad60e739b9dbb97f08a3623db961a74682422a |
[production] |
| 20:30 |
<bblack> |
lead.wikimedia.org: replaced by cobalt functionally, please leave it untouched for now with puppet disabled! |
[production] |
| 20:23 |
<twentyafterfour> |
restarted phd on iridium |
[production] |
| 20:08 |
<twentyafterfour> |
disabled phd + puppet on iridium and scheduled downtime in icinga to silence alerts |
[production] |
| 19:46 |
<mutante> |
deleted old /var/lib/gerrit2/ data on cobalt, syncing from lead |
[production] |
| 19:45 |
<mutante> |
rsyncing /var/lib/gerrit2 from lead to cobalt |
[production] |
| 19:30 |
<mutante> |
removed gerrit IPs from cobalt interfaces |
[production] |
| 19:29 |
<mutante> |
disabled puppet on lead and cobalt |
[production] |
| 19:21 |
<mutante> |
re-enabling puppet on cobalt |
[production] |
| 19:21 |
<mutante> |
removed gerrit IP from lead's interface, v4 and v6 |
[production] |
| 19:09 |
<mutante> |
rsyncing gerrit data one more time from lead to cobalt |
[production] |
| 19:08 |
<cmjohnson1> |
db1065 swapping failed disk slot 9 T147396 |
[production] |
| 19:07 |
<ostriches> |
stopped puppet on lead |
[production] |
| 19:07 |
<mutante> |
stopping gerrit on lead |
[production] |
| 19:02 |
<mutante> |
cobalt, disabled puppet, removed service IP from interface |
[production] |
| 17:28 |
<mutante> |
rsyncing gerrit data from lead to cobalt |
[production] |
| 16:53 |
<jynus> |
testing img_metadata nuking for T145953 and T147015 (backups on neodymium) |
[production] |
| 16:25 |
<gehel> |
reimage maps-test2001 - T147194 |
[production] |
| 16:14 |
<akosiaris> |
build python-irclib for jessie and upload it to apt.wikimedia.org jessie-wikimedia/main |
[production] |
| 16:06 |
<moritzm> |
updated hhvm package for jessie to 3.12.9 |
[production] |
| 14:41 |
<moritzm> |
uploaded openssl 1.0.2j for jessie-wikimedia to carbon |
[production] |
| 14:41 |
<kart_> |
Update cxserver to fa2f715 (T147552) |
[production] |
| 13:56 |
<elukey> |
reimaing mw123[45] to Debian Jessie (last two api appservers) |
[production] |
| 12:39 |
<elukey> |
reimaging mw123[23] to Debian Jessie |
[production] |
| 12:09 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1082 with its original weight - T145533 (duration: 00m 52s) |
[production] |
| 11:17 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1082 with a bit less weight than usual to start with - T145533 (duration: 00m 55s) |
[production] |
| 10:52 |
<moritzm> |
reimaging mw1238, mw1239 to jessie |
[production] |
| 10:28 |
<elukey> |
reimaging mw123[01] to Debian Jessie |
[production] |
| 10:27 |
<elukey> |
mw122[89] back in live api server pool |
[production] |
| 10:00 |
<_joe_> |
updated conftool to 0.3.1 on all the cluster except caches, T147480 |
[production] |
| 09:48 |
<_joe_> |
creating etcd100[1-6].eqiad.wmnet on ganeti, T147620 |
[production] |
| 09:32 |
<moritzm> |
reimaging mw1220, mw1236, mw1237 to jessie |
[production] |
| 09:28 |
<moritzm> |
installing pillow/python-imaging security updates on Ubuntu systems |
[production] |
| 09:20 |
<gehel> |
reimaging maps-test2004 - T147194 |
[production] |
| 09:18 |
<moritzm> |
installing php security updates on precise hosts |
[production] |
| 08:53 |
<gehel> |
reimaging maps-test2003 - T147194 |
[production] |
| 08:33 |
<oblivian@puppetmaster1001> |
conftool action : set/weight=20; selector: cluster=api_appserver,dc=eqiad,name=mw123.* |
[production] |
| 08:31 |
<oblivian@puppetmaster1001> |
conftool action : set/weight=0; selector: cluster=api_appserver,dc=eqiad,name=mw123.* |
[production] |
| 08:20 |
<_joe_> |
restarting hhvm on a few api appservers, due to memory leaks (T146451) |
[production] |
| 07:17 |
<elukey> |
reimaging mw1228 and mw1229 (api appservers) to Debian Jessie |
[production] |
| 06:32 |
<moritzm> |
reimaging mw1216, mw1218, mw1219 to jessie |
[production] |
| 06:31 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1082 to get its raid controller firmware upgraded - T145533 (duration: 00m 49s) |
[production] |
| 06:10 |
<_joe_> |
restarted hhvm, jobrunner on mw1161 |
[production] |
| 05:55 |
<marostegui> |
Deploying schema change on S4 master commonswiki.revision table - T147113 |
[production] |