|
2019-01-31
§
|
| 13:19 |
<mvolz@deploy1001> |
scap-helm zotero upgrade staging -f zotero-values-staging.yaml --version=0.0.1 stable/zotero [namespace: zotero, clusters: staging] |
[production] |
| 13:18 |
<mvolz@deploy1001> |
scap-helm zotero upgrade staging -f zotero-values-staging.yaml stable/zotero [namespace: zotero, clusters: staging] |
[production] |
| 12:54 |
<jynus> |
stop, upgrade and restart db2044 |
[production] |
| 12:12 |
<jynus> |
apply new grants to m5-master with replication T214740 |
[production] |
| 11:30 |
<arturo> |
T215012 icinga downtime cloudvirt1015 for 4h while investigating issues |
[production] |
| 11:24 |
<arturo> |
T215012 reboot cloudvirt1015 |
[production] |
| 11:24 |
<jynus> |
restart eventstreams on scb1002,3,4 |
[production] |
| 11:22 |
<jynus> |
restart eventstreams on scb1001 |
[production] |
| 10:22 |
<jynus> |
resetting to defaults innodb consistency options for db2048 T188327 |
[production] |
| 10:00 |
<jynus> |
restarting pdfrender on scb1002,3,4 |
[production] |
| 09:54 |
<jynus> |
restarting pdfrender on scb1001 |
[production] |
| 02:01 |
<gtirloni> |
T215004 restarted gerrit (using 1200% cpu, 71% mem) |
[production] |
|
2019-01-30
§
|
| 20:28 |
<bawolff_> |
reset 2FA@wikitech for [[User:deigo]] |
[production] |
| 18:25 |
<ladsgroup@deploy1001> |
Finished deploy [ores/deploy@ad160b0]: (no justification provided) (duration: 12m 46s) |
[production] |
| 18:12 |
<ladsgroup@deploy1001> |
Started deploy [ores/deploy@ad160b0]: (no justification provided) |
[production] |
| 18:03 |
<jynus> |
reducing innodb consistency options for db2048 T188327 |
[production] |
| 17:36 |
<XioNoX> |
deactivate/activate cr2-esams:xe-0/1/3 |
[production] |
| 17:28 |
<akosiaris> |
restart pdfrender on scb1003, scb1004 |
[production] |
| 16:19 |
<akosiaris> |
restart proton on proton1002 |
[production] |
| 15:52 |
<jynus> |
stop, upgrade and restart db2037 |
[production] |
| 15:24 |
<jynus> |
stop, upgrade and restart db2042 |
[production] |
| 14:27 |
<jynus> |
stop, upgrade and restart db2034, this will cause some lag on x1-codfw |
[production] |
| 13:53 |
<jynus> |
stop, upgrade and restart db2069 |
[production] |
| 11:20 |
<jynus> |
stop, upgrade and restart db2045, this will cause some lag on s8-codfw |
[production] |
| 10:54 |
<jynus> |
stop, upgrade and restart db2079 |
[production] |
| 10:33 |
<jynus> |
stop, upgrade and restart db2039, this will cause some lag on s6-codfw |
[production] |
| 10:03 |
<jynus> |
stop, upgrade and restart db2052, this will cause some lag on s5-codfw |
[production] |
| 09:31 |
<jynus> |
stop, upgrade and restart db2089 (s5/s6) |
[production] |
| 08:58 |
<jynus> |
stop, upgrade and restart db2051, this will cause some lag on s4-codfw |
[production] |
| 08:44 |
<jynus> |
stop, upgrade and restart db2090 |
[production] |
|
2019-01-29
§
|
| 21:52 |
<jijiki> |
Depooling thumbor2002 due to disc failure - T214813 |
[production] |
| 16:51 |
<arturo> |
T214499 update Netbox status for cloudvirt1023/1024/1025/1026/1027 from PLANNED to ACTIVE. These servers are actually providing services already. |
[production] |
| 10:05 |
<jynus> |
stop, upgrade and restart db2065 |
[production] |
| 09:28 |
<jynus> |
stop, upgrade and restart db2058 |
[production] |
| 09:12 |
<jynus> |
stopping, upgrading and restarting db2035, this will cause lag on codfw-s2 |
[production] |
| 08:58 |
<jynus> |
stop, upgrade and restart db2041 |
[production] |
| 08:38 |
<jynus> |
stop, upgrade and restart db2056 |
[production] |
| 08:17 |
<jynus@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1114 after crash (duration: 00m 52s) |
[production] |
| 03:32 |
<XioNoX> |
bump cr2-esams-cr2-eqiad ospf cost to 2000 for level3 link flapping |
[production] |
|
2019-01-28
§
|
| 23:51 |
<vgutierrez> |
restarting cp2014 - T214872 |
[production] |
| 21:02 |
<Zoranzoki21> |
Done wikitext export of content of database for education program on srwiki - T174802 (duration: 8 minutes) |
[production] |
| 20:54 |
<Zoranzoki21> |
Starting wikitext export of content of database for education program on srwiki - T174802 (21:54 UTC+1) |
[production] |
| 19:55 |
<brion> |
running final pass of requeueTranscodes.php on all wikis to make sure stray missing VP9 transcodes are cleaned up |
[production] |
| 16:41 |
<hashar> |
contint1001: cleaning up disk space on / (docker images) |
[production] |
| 16:36 |
<jynus> |
remove backups dir at dbstore2001 T214831 |
[production] |
| 15:22 |
<thcipriani> |
restarting jenkins for update |
[production] |
| 14:16 |
<jynus> |
stop, upgrade and reboot db2048, this will cause general lag/read only on enwiki/s1-codfw for some minutes |
[production] |
| 13:52 |
<jynus> |
stop, upgrade and reboot db2092 |
[production] |
| 12:55 |
<jynus> |
stop, upgrade and reboot db2085 |
[production] |
| 12:45 |
<jynus> |
powercycle ms-be1034 |
[production] |