Recent changes
Unfortunately, grouping results by page is not available on mobile. If you would like to use the group by page feature please visit the desktop site.
Track the most recent changes to the wiki on this page.
List of abbreviations:
- N
- This edit created a new page (also see list of new pages)
- m
- This is a minor edit
- b
- This edit was performed by a bot
- (±123)
- The page size changed by this number of bytes
10 May 2024
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 21:06 -1,324 Rfulcher talk contribs With Europa back online raid DIFFERENCE=.9560
- ITP:Act/20151208.23:57/20220110-lfs-us-moodle-LoopBacks 21:02 +1,348 Rfulcher talk contribs →202404 Daily RSync 2015-2024 and DigiPen Moodle: : 05/10/24, 03:04 - [it-alerts] cron wwwprod89 distancersync 20240510.0015 20240510.0304 50G:96% 96G:91% 126G:90% 420G:70% 131G:83% MoodleSG2024 619G 131G 83% : 05/10/24, 07:40 - [it-alerts] cron wwwprod27 distancersync 20240510.0015 20240510.0740 421G:92% 111G:95% 575G:81% 328G:88% 604G:83% 335G:83% Moodle2024 1.6T 335G 83%
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 20:58 +3,599 Rfulcher talk contribs Need to create the new sw98us and sw99us Port-Channel23 and change E23 to channel-group 23 mode active Finished and new all SATA mfsbrick0400 RAID-5 64T back into service.
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 19:46 +8,270 Rfulcher talk contribs all SATA mfsbrick0400 RAID-5 64T array initialized and ready to go back into service. Still need to reconfigure the SFP as an MLAG+LACP Channel
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 18:37 +635 Rfulcher talk contribs rearranged headings.
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 18:28 +1,109 Rfulcher talk contribs Exported and deleted tw_cli /c0/u0 Neptune:mfsbrick0700 27939.6GB RAID-5 volume # Export and delete Europa:mfsbrick0400
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 18:18 -49,664 Rfulcher talk contribs Prune progress.
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 18:13 +50,867 Rfulcher talk contribs Progress. ::* 20240510 ~1500 Neptune disk reads appear to have reached zero, replication of marked for removal data should be complete now?
- ITP:Act/20161026.02:11/20220718-imap01 imap01Archive03 17:56 +194 Rfulcher talk contribs Do another imap01Archive04/saturn-SYS
- ITP:Act/20161026.02:11/20220718-imap01 imap01Archive03 17:50 +1,411 Rfulcher talk contribs : 05/10/24, 13:03 - [it-alerts] saturn SyncImapToLizardFS.sh 34%/ 45%local 22%archive 45%alumni 50%student 66%faculty 29%wanic 75%staff : 05/09/24, 21:40 - [it-alerts] imap01 CompressImapBackups.sh 4.0T 322G 93% /zfs/imap01Archive04
- ITTR:Spam UCE Policy 02:01 +1,760 Ssulaimi talk contribs Im having trouble with whitelisting tricorglobal.com
9 May 2024
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 20:30 -54,903 Rfulcher talk contribs Prune the replication progress lines. ::* Thu May 9 18:42:17 PDT erinome(99.00%) is now full and not longer writing data. Neptune disk read rate is about the same, so Jupiter and Pluto are writing slightly more data now.
- ITTR:US Servers/Faculty/UniTime2 20:23 +12,053 Rfulcher talk contribs conformed unitime3 upgraded to java-17-oracle
- ITTR:US Servers/Faculty/UniTime2 19:47 +11,691 Rfulcher talk contribs Beginning ====== 20240509 Upgrade to java version 17 ====== did UpUniTime3From2 and confirmed still working. Halted, snapshot of unitime3 "PreJava17" boot back up, resume queueprocessor manually.
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 17:52 +45,367 Rfulcher talk contribs :* CS Charts do still show disk reads from Neptune remain high at ~36~48MBps as it continues to replicate these 27k chunks, erinome will be full soon and should stop taking on chunks.
- ITP:Act/20161026.02:11/20220718-imap01 imap01Archive03 17:41 +2,258 Rfulcher talk contribs :: Thu May 9 17:28:53 PDT 2024 finished rsync to /zfs/imap01Archive04/saturn-SYS.rsync.bak which began 20240509.172052 # another backup Add more monthly headings.
- ITFAQ:Workstation Hardware/Redmond On Campus Specifications 14:03 -30 Tabatha.tipton talk contribs →CLASSROOMS
- ITFAQ:Workstation Hardware/Redmond On Campus Specifications 13:54 0 Tabatha.tipton talk contribs →CLASSROOMS
- ITTR:Powershell 00:08 +581 Ssulaimi talk contribs
8 May 2024
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 22:42 +12,077 Rfulcher talk contribs progress
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 17:19 +10,479 Rfulcher talk contribs US_XCPNG82_Pool wb00001 Zab00 finally finished, cleaning up goals and dropping from raid2 to raid1
- ITP:Act/20161026.02:11/20220718-imap01 imap01Archive03 17:16 +974 Rfulcher talk contribs New rfulcher@imap01.digipen.edu:/zfs/imap01Archive04/saturn-SYS backup
- ITP:Act/20161026.02:11/20220718-imap01 imap01Archive03 17:05 +745 Rfulcher talk contribs John Watkins Modified imapd.conf on Saturn to disable tls 1.0, Cyrus did not add support for TLS v1.3 until 3.0.12 and we still run 2.4.21
- ITP:Act/20160512.20:55/2024 lfs-isr1 XenClient XenBatch Zab00 16:14 +1,011 Rfulcher talk contribs === 20240503 XenBatch Zab00 Cron === .... .... Record each pools email report subject.
7 May 2024
- ITP:Act/20151208.23:57/20180706-Migrate Perforce into LoopBack 20:20 +9,131 Rfulcher talk contribs Same error every one minute. We must be missing a way to exchange this new replica server "server1" trust certificate?
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 18:59 +6,112 Rfulcher talk contribs replication progress Weekend Zab is still in progress for US_XCPNG82_Pool wb00001
- m ITP:Act/20151208.23:57/20180706-Migrate Perforce into LoopBack 18:24 -2 Rfulcher talk contribs fix heading depth
- ITP:Act/20151208.23:57/20180706-Migrate Perforce into LoopBack 18:23 +3,520 Rfulcher talk contribs More work on ==== 20240418 broken sinless_theory svn team ==== move the journals and checkpoints to network storage so local database is not on same storage like we did some time ago with perfoce3.
- ITP:Act/20161026.02:11/20220718-imap01 imap01Archive03 17:03 +207 Rfulcher talk contribs : 05/07/24, 12:06 - [it-alerts] saturn SyncImapToLizardFS.sh 33%/ 45%local 22%archive 45%alumni 50%student 66%faculty 29%wanic 75%staff : 05/06/24, 20:59 - [it-alerts] imap01 CompressImapBackups.sh 4.0T 334G 93% /zfs/imap01Archive04
- ITTR:MRBS Scheduling 16:53 +131 Rfulcher talk contribs →20240507 Auth Changes: * Johnny verbally clarified that this new user does not need administrative permissions for MRBS nor MachForms, just general user access which all users get.
- ITTR:MRBS Scheduling 16:27 +1,204 Rfulcher talk contribs New ==== 20240507 Auth Changes ==== and also Machforms changes.
- ITFAQ:Workstation Hardware/Redmond On Campus Specifications 11:53 +5 Tabatha.tipton talk contribs →CLASSROOMS
- ITFAQ:Workstation Hardware/Redmond On Campus Specifications 11:47 +840 Tabatha.tipton talk contribs →CLASSROOMS
- ITTR:Moodle/SG-2024Summer 00:51 +390 Pleilani talk contribs To add other languages in moodle for KOREA
6 May 2024
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 20:36 +10,367 Rfulcher talk contribs :* At about 19:30 replication is finally resuming
- ITP:Act/20151208.23:57/20220110-lfs-us-moodle-LoopBacks 18:49 +1,689 Rfulcher talk contribs →202404 Daily RSync 2015-2024 and DigiPen Moodle: : 05/06/24, 04:04 - [it-alerts] cron wwwprod89 distancersync 20240506.0015 20240506.0404 50G:96% 96G:91% 126G:90% 419G:70% 136G:82% MoodleSG2024 613G 136G 82% : 05/06/24, 06:49 - [it-alerts] cron wwwprod27 distancersync 20240506.0015 20240506.0649 421G:92% 111G:95% 575G:81% 328G:88% 610G:82% 346G:82% Moodle2024 1.6T 346G 82%
- ITP:Act/20161026.02:11/20220718-imap01 imap01Archive03 18:42 +566 Rfulcher talk contribs : 05/06/24, 11:18 - [it-alerts] saturn SyncImapToLizardFS.sh 33%/ 45%local 22%archive 45%alumni 50%student 66%faculty 29%wanic 75%staff : 05/05/24, 20:55 - [it-alerts] imap01 CompressImapBackups.sh 4.0T 334G 93% /zfs/imap01Archive04
- ITTR:Backup/Bacula/Audits 18:31 +116 Rfulcher talk contribs Incr07CQ - ITTR:Backup/Bacula/Audits/US/2024-05-03-Incr - 0112.0725 (130045 Sec) - 2727465778572 (2.73 TB) - 0111.2803 (127683 Sec) - DP0177+DP0176+DP0175
- diffhistN ITTR:Backup/Bacula/Audits/US/2024-05-03-Incr 18:27 +84,600 Rfulcher talk contribs Incr07CQ - 0112.0725 (130045 Sec) - 2727465778572 (2.73 TB) - 0111.2803 (127683 Sec) - DP0177+DP0176+DP0175
- ITP:Act/20140425.19:10/20240312 202405Summer Prep 17:04 +1,613 Rfulcher talk contribs setup === Enable VPL Server Settings === * Applied the following on 2024.05.06 https://distance.digipen.edu/2024-summer/admin/settings.php?section=modsettingvpl
- ITFAQ:Workstation Hardware/Redmond On Campus Specifications 14:55 +6 Tabatha.tipton talk contribs →CLASSROOMS
- ITFAQ:Workstation Hardware/Redmond On Campus Specifications 14:52 +18 Tabatha.tipton talk contribs →CLASSROOMS
- ITFAQ:Workstation Hardware/Redmond On Campus Specifications 14:50 +1 Tabatha.tipton talk contribs →CLASSROOMS
- ITFAQ:Workstation Hardware/Redmond On Campus Specifications 14:49 -3 Tabatha.tipton talk contribs →CLASSROOMS
- ITFAQ:Workstation Hardware/Redmond On Campus Specifications 14:48 -152 Tabatha.tipton talk contribs →CLASSROOMS
- ITFAQ:Workstation Hardware/Redmond On Campus Specifications 14:45 +114 Tabatha.tipton talk contribs →CLASSROOMS
- ITFAQ:Workstation Hardware/Redmond On Campus Specifications 14:35 +1,254 Tabatha.tipton talk contribs →CLASSROOMS
5 May 2024
- ITTR:Moodle/SG-2024Summer 22:47 +325 Pleilani talk contribs add a new moodle checklist. need to set default timezone to Asia/Singapore
3 May 2024
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 20:01 -2,660 Rfulcher talk contribs :: Fri May 3 20:00:26 PDT 2024 lfs-us-isr1-03 Current raid DIFFERENCE=0 ACCEPTABLE_DIFFERENCE = 0.01 # undergoal (0) / - stable (1393350) / - overgoal (114396) / - pending deletion (26) /
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 19:57 +2,658 Rfulcher talk contribs Some additional Neptune marked for removal progress.
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 18:53 +4,766 Rfulcher talk contribs New ==== Mark for Removal Neptune early so we can give the 10T SATA disks to Europa which can mix SAS disks==== replication has begun.
- ITP:Act/20160512.20:55/2024 lfs-isr1 XenClient XenBatch Zab00 16:34 +712 Rfulcher talk contribs === 20240426 XenBatch Zab02 Cron === .... .... Record each pools email report subject.
- ITP:Act/20161026.02:11/20220718-imap01 imap01Archive03 16:29 +1,166 Rfulcher talk contribs : 05/03/24, 13:17 - [it-alerts] saturn SyncImapToLizardFS.sh 33%/ 45%local 22%archive 45%alumni 50%student 66%faculty 29%wanic 75%staff : 05/02/24, 21:09 - [it-alerts] imap01 CompressImapBackups.sh 4.0T 344G 93% /zfs/imap01Archive04
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 15:52 +7,056 Rfulcher talk contribs NOTE, Neptune has 4*SATA 10TiB Disks but the controller might not allow for mixed SATA+SAS Array the way that Eurpoa does. * '''Might want to retire Neptune now, and rebuild Europa at an 8*10TiB SATA since it appears that it will allow for SAS member replacements upon the failure of the older weak SATA members.'''
- ITTR:Singapore/Semester Checklist 01:13 +909 Pleilani talk contribs Initialize Summer 2024 checklist.
1 May 2024
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 21:13 +10,827 Rfulcher talk contribs Rebuild Europa metasas array mfsbrick0400 with old Arche mfsbrick.02.12 mfsbrick.02.17 mfsbrick.02.20 disks is working except for smart data from the ExArche SAS verses originial SATA disks.
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 19:00 +8,392 Rfulcher talk contribs lfs-us-isr1 DIFFERENCE=.1389 # John Watkins Bouncing neptune & pluto & ananke for updates. # Check on more mfsbrick-lfs-us-courses-Apr_25 and still did not find goal=1 orphaned chunks # arche temperature weird 60 value is gone now.
- ITTR:Backup/Bacula/Audits 17:01 +201 Rfulcher talk contribs Incr07CP - ITTR:Backup/Bacula/Audits/US/2024-04-26-Incr - 0106.5845 (111525 Sec) - 1460587682808 (1.46 TB) - 0106.1138 (108698 Sec) - DP0177 # ''(DP0178 Resume Error due to ceres upgrade and reboot wrote 664G or ~2000G, re-marked as Full)''
- diffhistN ITTR:Backup/Bacula/Audits/US/2024-04-26-Incr 16:59 +84,688 Rfulcher talk contribs Incr07CP - 0106.5845 (111525 Sec) - 1460587682808 (1.46 TB) - 0106.1138 (108698 Sec) - DP0177+DP0178 # ''(DP0178 Resume Error due to ceres upgrade and reboot wrote 664G or ~2000G, re-marked as Full)''
- ITFAQ:Workstation Hardware/Redmond On Campus Specifications 11:37 +11 Tabatha.tipton talk contribs →CLASSROOMS
30 April 2024
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 12:31 +1,629 Rfulcher talk contribs Europa mfsbrick0400 marked for removal finished, stopped CS while ACCEPTABLE_DIFFERENCE proceeds.
- ITP:Act/20151208.23:57/20220110-lfs-us-moodle-LoopBacks 12:11 +851 Rfulcher talk contribs →202404 Daily RSync 2015-2024 and DigiPen Moodle: : 04/30/24, 02:48 - [it-alerts] cron wwwprod89 distancersync 20240430.0015 20240430.0248 50G:96% 96G:91% 126G:90% 421G:70% 148G:81% MoodleSG2024 596G 148G 81% : 04/30/24, 06:49 - [it-alerts] cron wwwprod27 distancersync 20240429.0015 20240430.0649 422G:92% 111G:95% 575G:81% 328G:88% 553G:84% 66G:96% Moodle2024 1.4T 66G 96% Expanded US +400G
29 April 2024
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 22:11 +13,910 Rfulcher talk contribs Europa marked for removal progress: raid2 replication is needed 8633 / 372270 (97.69%)
- ITP:Act/20151208.23:57/20220110-lfs-us-moodle-LoopBacks 20:30 +1,254 Rfulcher talk contribs →202404 Daily RSync 2015-2024 and DigiPen Moodle: : 04/29/24, 02:44 - [it-alerts] cron wwwprod89 distancersync 20240429.0015 20240429.0244 50G:96% 96G:91% 126G:90% 421G:70% 149G:80% MoodleSG2024 596G 149G 80% : 04/27/24, 02:37 - already running - Ran Long... : 04/28/24, 09:20 - [it-alerts] cron wwwprod27 distancersync 20240427.0015 20240428.0919 422G:92% 111G:95% 575G:81% 328G:88% 539G:85% 83G:95% Moodle2024 1.4T 83G 95%
- ITP:Act/20161026.02:11/20220718-imap01 imap01Archive03 20:16 +842 Rfulcher talk contribs : 04/29/24, 11:16 - [it-alerts] saturn SyncImapToLizardFS.sh 33%/ 44%local 21%archive 45%alumni 49%student 66%faculty 29%wanic 75%staff : 04/28/24, 20:58 - [it-alerts] imap01 CompressImapBackups.sh 4.0T 346G 93% /zfs/imap01Archive04
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 16:51 +638 Rfulcher talk contribs :: Sat Apr 27 09:24:47 PDT 2024 Some jbods replication is needed 179 / 3003480 (100.0%) - 2024.04.29 16:44:59 PDT - amalthea:23975079:182/242TiB:75.03% - arche:23976197:216/271TiB:79.72%
26 April 2024
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 19:41 +485 Rfulcher talk contribs Europa Marked for Removal lfs-us-isr DIFFERENCE=.0098 # JBOD mfs-data 91.58%
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 18:11 +1,714 Rfulcher talk contribs Jupiter lfs-us-isr DIFFERENCE=.0098 Finished # JBOD mfs-data 88.12% # lfs-us-rcs,courses,moodle Finished # 20240426 Now that Jupiter:mfsbrick0300(62TiB) is back in service we can mark Europa:mfsbrick0400 for removal, soon.
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 17:51 +1,510 Rfulcher talk contribs More mfsbrick-lfs-us-courses-Apr_25 investigation
- ITFAQ:Workstation Hardware/Redmond On Campus Specifications 15:19 +28 Tabatha.tipton talk contribs →CLASSROOMS
- ITFAQ:Workstation Hardware/Redmond On Campus Specifications 08:52 0 Tabatha.tipton talk contribs
- ITFAQ:Workstation Hardware/Redmond On Campus Specifications 08:51 +113 Tabatha.tipton talk contribs
- ITFAQ:Workstation Hardware/Redmond On Campus Specifications 08:48 +176 Tabatha.tipton talk contribs
- ITFAQ:Workstation Hardware/Redmond On Campus Specifications 08:46 -245 Tabatha.tipton talk contribs →CLASSROOMS Tag: Visual edit: Switched
25 April 2024
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 20:17 +12,176 Rfulcher talk contribs Jupiter lfs-us-isr DIFFERENCE=.0534 # JBOD mfs-data 78.02% # Trying to track down orphaned jbod1 courses chunk owners.
- ITTR Talk:Sympa 19:02 +1,154 Rfulcher talk contribs Finished cleanup 60G 68% /data/services/sympa
- ITP:Act/20161026.02:11/20220718-imap01 imap01Archive03 18:50 +207 Rfulcher talk contribs : 04/25/24, 14:57 - [it-alerts] saturn SyncImapToLizardFS.sh 33%/ 44%local 21%archive 44%alumni 49%student 66%faculty 29%wanic 75%staff : 04/24/24, 21:14 - [it-alerts] imap01 CompressImapBackups.sh 4.0T 362G 92% /zfs/imap01Archive04
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 16:36 +606 Rfulcher talk contribs Jupiter lfs-us-isr DIFFERENCE=.0534 # JBOD mfs-data 74.57% # lfs-us-rcs,courses,moodle Finished
24 April 2024
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 22:09 +1,145 Rfulcher talk contribs * lfs-us-courses is missing 1 chunk on old mfsbrick.00.16 from that set of goal jbod1 that we can not identify # Perhaps bring back online which should let duplicate chunks get purged, then mark it for removal so the list chunk can get coppied elsewhere.
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 21:45 +293 Rfulcher talk contribs lfs-us-courses Finished _REP_LIMIT 1
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 19:14 +2,390 Rfulcher talk contribs An Arche sensor temp1 changed from 32 to 60 along with LED and audible alarm as of Apr 21 08:26:01
- ITP:Act/20160512.20:55/2024 lfs-isr1 XenClient XenBatch Zab00 18:52 +605 Rfulcher talk contribs New === 20240423 Periodic SGBounceMetaLoggers === 2961GB Freed up
- ITP:Act/20151208.23:57/20220110-lfs-us-moodle-LoopBacks 18:45 +1,214 Rfulcher talk contribs →202404 Daily RSync 2015-2024 and DigiPen Moodle: MoodleSG2024 593G 151G 80% # Moodle2024 1.3T 163G 90%
- ITP:Act/20161026.02:11/20220718-imap01 imap01Archive03 18:40 +207 Rfulcher talk contribs : 04/24/24, 13:48 - [it-alerts] saturn SyncImapToLizardFS.sh 32%/ 44%local 21%archive 44%alumni 49%student 66%faculty 29%wanic 75%staff : 04/23/24, 21:33 - [it-alerts] imap01 CompressImapBackups.sh 4.0T 364G 92% /zfs/imap01Archive04
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 18:37 +1,503 Rfulcher talk contribs Jupiter lfs-us-isr DIFFERENCE=.0914 # JBOD mfs-data 64.51% # lfs-us-rcs Finished # lfs-us-courses 96.24% # lfs-us-moodle Finished Apr 24 08:18:09
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 18:30 +4,320 Rfulcher talk contribs : Tabatha Tipton - 20240424 11:23 AM - Jacob, the houdini-20.0.547-win64-vc143.exe we have on dragons' den is broken. You download it and attempt to run it and windows gives the "this app cant run on your PC" error. I confirmed by downloading the file from the den and running it on my work laptop and it gives me the error as well. OTRSTicket:3064853 — Helpdesk Ticket (Michelle Wong)
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 17:58 +297 Rfulcher talk contribs * Got email reply from Bruce confirming he downloaded the .zip, he asked for moodle data, but I declined. I forwareded our exchange to: susan.grimm, b.maryott, asuka, gary.oyler, jdeek
23 April 2024
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 19:42 +1,115 Rfulcher talk contribs Tangent #3007078 bruce.dellinger Lost Work:
- ITTR Talk:Sympa 18:12 -4,321 Rfulcher talk contribs Prune progress to 6hour steps. :: Tue 23 Apr 2024 06:09:54 PM PDT 26G /mnt/LOSTFOUND.old/ # ~50% finished
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 18:07 +1,792 Rfulcher talk contribs Jupiter lfs-us-isr DIFFERENCE=.1504 # JBOD mfs-data 51.27% # lfs-us-rcs Finished # lfs-us-courses 68.49% # lfs-us-moodle 87.86%
- ITTR Talk:Sympa 17:52 +4,574 Rfulcher talk contribs sympa1 rsync data/services/sympa/LOSTFOUND.old to archives/sympa progress.
- ITP:Act/20240319.23:55 17:51 +312 Rfulcher talk contribs 4/18/24, 12:41 - 4/23/24, 12:13 - [it] ShoreWare Event Notification: Switch: (116) - x11 Warnings
- ITP:Act/20240319.23:55 17:48 +2,278 Rfulcher talk contribs 20240423 - Dactyl has been rebooting, Johnny cleaned up network interfaces.
- ITTR Talk:Storage/Bricks 17:41 +448 Rfulcher talk contribs europa:megas,2 pass2 ST10000VN0004-1ZD101 ZA26JQ8N SATA ReAlSec:2040
- ITP:Act/20161026.02:11/20220718-imap01 imap01Archive03 17:33 +207 Rfulcher talk contribs : 04/23/24, 15:10 - [it-alerts] saturn SyncImapToLizardFS.sh 32%/ 44%local 21%archive 44%alumni 49%student 66%faculty 29%wanic 75%staff : 04/22/24, 21:10 - [it-alerts] imap01 CompressImapBackups.sh 4.0T 363G 92% /zfs/imap01Archive04
- ITTR:Backup/Bacula/Audits 17:30 -2 Rfulcher talk contribs Cleanup up some Container 208108 notes to match the monthly emails
- ITP:Act/20150625.02:30/202101-Prometheus sm1083224 Blades ARP Metalogger 17:20 +10,142 Rfulcher talk contribs * Last was #20240423 metaloggers SGBounceMetaLoggers '''2961GB Freed up''' (307+1059+531+1064=2961GB)
- ITP:Act/20160512.20:55/2024 lfs-isr1 XenClient XenBatch Zab00 16:38 +117 Rfulcher talk contribs === 2025 Recording in new article === TODO: * See ITP:Act/20160512.20:55/2025 lfs-isr1 XenClient XenBatch Zab00
- ITP:Act/20160512.20:55/2023 lfs-isr1 XenClient XenBatch Zab00 16:38 -47,613 Rfulcher talk contribs Moved to ITP:Act/20160512.20:55/2024 lfs-isr1 XenClient XenBatch Zab00 into new article for new year.
- diffhistN ITP:Act/20160512.20:55/2024 lfs-isr1 XenClient XenBatch Zab00 16:37 +49,002 Rfulcher talk contribs Moved from ITP:Act/20160512.20:55/2023 lfs-isr1 XenClient XenBatch Zab00 into new article for new year.
- ITP:Act/20160512.20:55/2023 lfs-isr1 XenClient XenBatch Zab00 16:31 +3,729 Rfulcher talk contribs === 20240419 XenBatch Zab01 Cron === .... .... Record each pools email report subject.
22 April 2024
- ITTR:Backup/Bacula/Audits 19:28 +146 Rfulcher talk contribs * Container 208109 - Incr07CC-Incr07CQ (20240125~20240503) DP0192-DP0177
- ITTR:Backup/Bacula/Audits 19:17 +1,475 Rfulcher talk contribs Incr07CO - ITTR:Backup/Bacula/Audits/US/2024-04-19-Incr - 0118.5729 (154649 Sec) - 1678644222342 (1.68 TB) - 0118.2334 (152614 Sec) - DP0179+DP0178
- diffhistN ITTR:Backup/Bacula/Audits/US/2024-04-19-Incr 19:05 +84,982 Rfulcher talk contribs Incr07CO - 0118.5729 (154649 Sec) - 1678644222342 (1.68 TB) - 0118.2334 (152614 Sec) - DP0179+DP0178
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 18:37 +378 Rfulcher talk contribs lfs-us-courses7 _REP_LIMIT =2 Expect ~16MBps
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 18:23 +2,163 Rfulcher talk contribs Jupiter lfs-us-isr DIFFERENCE=.1960 # JBOD mfs-data 38.19% (fixed a missing chunk in fg300.log-20240415-2215.gz) # lfs-us-rcs 88.68% # lfs-us-courses 41.28% # lfs-us-moodle 67.45%
- ITTR Talk:Sympa 18:04 +835 Rfulcher talk contribs New === 20240422 Stash some data sympa/LOSTFOUND.old to network archives ===
- ITTR Talk:Storage/Bricks 17:03 +2,212 Rfulcher talk contribs europa:megas,2 pass2 ST10000VN0004-1ZD101 ZA26JQ8N SATA ReAlSec:2008
- ITP:Act/20161026.02:11/20220718-imap01 imap01Archive03 16:53 +871 Rfulcher talk contribs : 04/22/24, 13:47 - [it-alerts] saturn SyncImapToLizardFS.sh 32%/ 44%local 21%archive 44%alumni 49%student 66%faculty 29%wanic 75%staff : 04/21/24, 20:59 - [it-alerts] imap01 CompressImapBackups.sh 4.0T 361G 92% /zfs/imap01Archive04
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 16:27 +5,822 Rfulcher talk contribs Jupiter lfs-us-isr DIFFERENCE=.2061 # JBOD mfs-data 34.86% # lfs-us-rcs 81.10% # lfs-us-courses 37.73% # lfs-us-moodle 62.24% # lfs-data-ssd Finished Apr 22 07:59:19
20 April 2024
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 01:08 +753 Rfulcher talk contribs DIFFERENCE=.2423 (2.570%) (5.930%) (2.690%) (6.120%) (9.930%)
- ITP:Act/20151208.23:57/20220110-lfs-us-moodle-LoopBacks 01:01 +457 Rfulcher talk contribs →202404 Daily RSync 2015-2024 and DigiPen Moodle: : 04/19/24, Ran Long... : 04/20/24, 00:57 - [it-alerts] cron wwwprod27 distancersync 20240419.0015 20240420.0057 374G:93% 111G:95% 575G:81% 328G:88% 498G:86% 50G:97% Moodle2024 1.3T 50G 97% : <code>Sat Apr 20 00:57:23 root@wwwprod27:~# NewLoop 1400 /lfs-us-isr1/LoopBacks/moodle/2024.zfs # 1200 1288490188800</code> +200GB Moodle2024 1.3T 1.3T 50G 97%
19 April 2024
- ITP:Act/20130612.00:19/2020-Sideloaded Arcade Games 20:41 +1,546 Rfulcher talk contribs 20240414 Kilonova_Setup.exe
- ITP:Act/20151208.23:57/20220110-lfs-us-moodle-LoopBacks 20:10 +649 Rfulcher talk contribs →202404 Daily RSync 2015-2024 and DigiPen Moodle: : 04/19/24, 02:41 - [it-alerts] cron wwwprod89 distancersync 20240419.0015 20240419.0241 50G:96% 96G:91% 126G:90% 417G:71% 154G:80% MoodleSG2024 590G 154G 80% : 04/18/24, None :: Fri Apr 19 20:09:06 root@wwwprod27:~# Moodle2024 1.3T 1.2T 64G 96% # Will need to expand soon.
- ITTR Talk:Storage/Bricks 20:03 +400 Rfulcher talk contribs * 2024-04-19T20:00:04 ::: Amalthea New Disks :: mfsbrick0010 /dev/da5 ST24000NM007H ZYD04AKL :: mfsbrick0011 /dev/da8 ST24000NM007H ZYD057T6 :: mfsbrick0016 /dev/da9 ST24000NM007H ZYD06NSL
- ITP:Act/20161026.02:11/20220718-imap01 imap01Archive03 19:59 +414 Rfulcher talk contribs : 04/19/24, 14:16 - [it-alerts] saturn SyncImapToLizardFS.sh 32%/ 44%local 21%archive 44%alumni 49%student 66%faculty 29%wanic 75%staff : 04/18/24, 21:21 - [it-alerts] imap01 CompressImapBackups.sh 4.0T 366G 92% /zfs/imap01Archive04
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 19:57 +694 Rfulcher talk contribs Would like to take these last three JBODfront Slots 21,22,23 offline at some point, change goal to move data elsewhere, if we have ~31+TiB Available at Amalthea:
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 19:39 +6,962 Rfulcher talk contribs lfs-us-isr1 DIFFERENCE=.2548 # mfs-data 3000682 (.1000%) # rcs 1663084 (.2300%) # courses 552725 (.0300%) # moodle 382539 (.9600%) # lfsx202 157529 (.1200%) # Three New Amalthea 24TiB Bricks swapped into service.
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 18:05 +1,763 Rfulcher talk contribs lfs-us-isr1 DIFFERENCE=.2672 # : John Watkins - 4:47 PM - amalthea needs a reboot to upgrade to 13.3
- ITTR Talk:Storage/Bricks 18:03 +193 Rfulcher talk contribs * 2024-04-19T18:00:04 ::: FreeBSD amalthea.digipen.edu 13.2-RELEASE-p4 upgrade to 13.3-RELEASE-p1 amd64 ::: mfsbrick0015 /dev/da22 became /dev/da21 ::: mfsbrick0019 /dev/da23 became /dev/da22
- ITTR Talk:Storage/Bricks 18:00 +1,338 Rfulcher talk contribs More minor pluto:3ware,3 ST4000DM000-1F2168 Z300JVLK and europa:megas,2 pass2 ST10000VN0004-1ZD101 ZA26JQ8N ReAlSec:1768
- ITP:Act/20151208.23:57/20180706-Migrate Perforce into LoopBack 14:52 +1,860 Rfulcher talk contribs →2024 svn errors: * According to trac top is rev1585 * Dumped revision 1582. (Was the top rev when the dump was started.) :: real 383m57.768s :: user 26m16.132s :: sys 8m27.260s
18 April 2024
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 20:53 +3,036 Rfulcher talk contribs Prep for ==== Amalthea swap out three 10TB for three 24TB Disks ==== to begin tomorrow.
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 20:36 +7,640 Rfulcher talk contribs Arche HUD * It appears that we have gained additional CPU or System temperatures? Realligning the old/new looks like this:
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 20:24 +8,289 Rfulcher talk contribs New ==== 20240418 Arche Reformat retired 10TiB Disks for use in Europa or other older Storage Servers ====
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 18:34 +6,583 Rfulcher talk contribs New ==== Arche Try more shorter duration rebalancing to see if mfsbrick0210 causes more crashing ==== not quite working as expected, rcs and mfs-data get more endangered than there should be which show up after a restart.
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 16:47 +1,830 Rfulcher talk contribs lfs-us-isr1 DIFFERENCE=.3182 # mfs-data finally finished scan+purge of mfsbrick0202
- ITP:Act/20151208.23:57/20180706-Migrate Perforce into LoopBack 16:26 +5,510 Rfulcher talk contribs →20240314 tangent broken s24_sophomoregame_team03 svn team: New transient ==== 20240418 broken sinless_theory svn team ==== commit problem.
- ITP:Act/20151208.23:57/20180706-Migrate Perforce into LoopBack 15:32 -4 Rfulcher talk contribs →20240314 tangent broken s24_sophomoregame_team03 svn team: drop heading depth
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 10:12 +2,896 Rfulcher talk contribs :: Thu 18 Apr 2024 09:42:09 AM PDT CS crashed 1125348 Apr 18 09:41:58 arche mfschunkserver[24997]: scanning folder /mfsbrick0210/mfs-data/: 99% (570s) : Apr 18 09:53:25 arche mfschunkserver[28076]: scanning folder /mfsbrick0202/mfs-data/: complete (669s) and finally did not crash! * 2024.04.18 10:11:18 PDT - endangered (26455) /
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 07:02 +7,530 Rfulcher talk contribs :: Thu 18 Apr 2024 06:55:22 AM PDT CS crashed 1126205 Apr 18 06:55:14 arche mfschunkserver[5445]: scanning folder /mfsbrick0210/mfs-data/: 96% (519s)
17 April 2024
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 23:35 +3,169 Rfulcher talk contribs :: Wed 17 Apr 2024 11:29:51 PM PDT CS crashed 1128788 Apr 17 23:29:33 arche mfschunkserver[16989]: scanning folder /mfsbrick0210/mfs-data/: 83% (494s)
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 20:38 +1,656 Rfulcher talk contribs :: Wed 17 Apr 2024 08:32:16 PM PDT CS crashed 1130165 Apr 17 20:31:59 arche mfschunkserver[18614]: scanning folder /mfsbrick0210/mfs-data/: 76% (409s)
- m ITTR:Reports/rfulcher/2022.10.21 19:52 0 Rfulcher talk contribs
- ITP:Act/20151208.23:57/20220110-lfs-us-moodle-LoopBacks 19:39 +729 Rfulcher talk contribs →202404 Daily RSync 2015-2024 and DigiPen Moodle: : 04/17/24, 03:16 - [it-alerts] cron wwwprod89 distancersync 20240417.0015 20240417.0316 50G:96% 96G:91% 126G:90% 416G:71% 154G:80% MoodleSG2024 589G 154G 80% : 04/17/24, 14:53 - [it-alerts] cron wwwprod27 distancersync 20240417.0015 20240417.1453 374G:93% 111G:95% 575G:81% 329G:88% 497G:86% 159G:88% Moodle2024 1.1T 159G 88%
- ITP:Act/20161026.02:11/20220718-imap01 imap01Archive03 19:33 +231 Rfulcher talk contribs : 04/17/24, 09:34 - [it-alerts] saturn SyncImapToLizardFS.sh 32%/ 44%local 21%archive 44%alumni 49%student 66%faculty 29%wanic 75%staff : 04/16/24, 21:31 - [it-alerts] imap01 CompressImapBackups.sh 4.0T 365G 92% /zfs/imap01Archive04
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 19:05 +9,575 Rfulcher talk contribs :: Wed 17 Apr 2024 07:04:01 PM PDT CS crashed 1130466 Apr 17 19:03:46 arche mfschunkserver[1383]: scanning folder /mfsbrick0210/mfs-data/: 75% (422s) # Tried tuning vfs_cache_pressure but did not speed up brick scanning.
- ITTR Talk:Storage/Bricks 16:32 +2,762 Rfulcher talk contribs Arche upgraded from 3.2.0-5-amd64 x86_64 Debian GNU/Linux 7.11 (wheezy) to 8.11 (jessie) to 9.13 (stretch) to 10 (buster) and 5.10.0-0.deb10.26-amd64 x86_64 # More of same recent europa:megas,2 pass2 ST10000VN0004-1ZD101 ZA26JQ8N SATA ReAlSec:1672
- ITTR:US Servers/Faculty/UniTime2 15:58 +1,893 Rfulcher talk contribs New ===== 20240417 Need newer java version 17 =====
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 13:36 +5,871 Rfulcher talk contribs :: Wed 17 Apr 2024 01:30:24 PM PDT CS crashed 1132624 Apr 17 13:30:13 arche mfschunkserver[1041]: scanning folder /mfsbrick0210/mfs-data/: 64% (340s)
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 09:51 +16,720 Rfulcher talk contribs :: Wed 17 Apr 2024 09:46:44 AM PDT CS crashed 1134188 Apr 17 09:46:30 arche mfschunkserver[4871]: scanning folder /mfsbrick0210/mfs-data/: 56% (312s)
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 01:36 +13,109 Rfulcher talk contribs :: Wed 17 Apr 2024 01:35:15 AM PDT CS crashed 1138373 Apr 17 01:35:06 arche mfschunkserver[21063]: scanning folder /mfsbrick0210/mfs-data/: 34% (182s)
16 April 2024
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 21:49 +3,165 Rfulcher talk contribs Restart loop but log each time it crashes:
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 21:24 +2,342 Rfulcher talk contribs Is slowly making progress, but still crashing when a mfsbrick0210 duplicate chunk is encountered for unknown reasons.
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 21:06 +17,170 Rfulcher talk contribs New ==== Arche mfs-data CS is crashing while bringing old bricks back online with reload ==== heading # mfsbrick0210 appears to be the brick causing the crash
- ITP:Act/20161026.02:11/20220718-imap01 imap01Archive03 19:24 +1,035 Rfulcher talk contribs : 04/16/24, 16:14 - [it-alerts] saturn SyncImapToLizardFS.sh 31%/ 44%local 21%archive 44%alumni 49%student 66%faculty 29%wanic 75%staff : 04/15/24, 21:23 - [it-alerts] imap01 CompressImapBackups.sh 4.0T 366G 92% /zfs/imap01Archive04
- ITP:Act/20160512.20:55/2023 lfs-isr1 XenClient XenBatch Zab00 18:45 +1,041 Rfulcher talk contribs === 20240412 XenBatch Zab00 Cron === .... .... Record each pools email report subject.
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 17:27 +16,301 Rfulcher talk contribs lfs-us-isr1 DIFFERENCE=.4264 # First ==== Arche try to rebalance until three 24TB disks use matches older Bricks ==== from 55% down to 50% difference # mfs-data is doing Segmentation fault when scanning old bricks, bringing online one at a time.
15 April 2024
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 22:32 +5,694 Rfulcher talk contribs Fix that update to Deb10 updated default mfschunkserver binary to v3.0.230 so reverted that back to 2.6.0. Start next 5% rebalance to new bricks that should take ~5ish hours.
- ITP:Act/20151208.23:57/20220110-lfs-us-moodle-LoopBacks 21:29 +3,603 Rfulcher talk contribs →202404 Daily RSync 2015-2024 and DigiPen Moodle: fix bad wwwprod89 MoodleSG2023 cache mountpoints were not empty.
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 20:34 +823 Rfulcher talk contribs Start First loop to reload once one additional percent at any new Brick is consumed.
- ITP:Act/20151208.23:57/20240212-More Raid Storage Refinements 20:16 +6,455 Rfulcher talk contribs lfs-us-isr1 DIFFERENCE=.4561 # mfs-data Finished # lfs-us-courses Finished. # Start new ==== Arche try to rebalance until three 24TB disks use matches older Bricks ==== heading.
- diffhistN ITTR:Backup/Bacula/Audits/US/2024-04-12-Incr 17:47 +84,814 Rfulcher talk contribs Incr07CN - 0111.2400 (127440 Sec) - 1363831245981 (1.36 TB) - 0110.5414 (125654 Sec) - DP0180+DP0179
- ITTR:Backup/Bacula/Audits 17:46 +815 Rfulcher talk contribs Incr07CN - ITTR:Backup/Bacula/Audits/US/2024-04-12-Incr - 0111.2400 (127440 Sec) - 1363831245981 (1.36 TB) - 0110.5414 (125654 Sec) - DP0180+DP0179
- ITP:Act/20160512.20:55/2023 lfs-isr1 XenClient XenBatch Zab00 16:28 +2,264 Rfulcher talk contribs : Last backup did manage to finish before the next weeks jobs: # Refine === Speed up slowest US_Colleague_Pool & US_XCPNG82_Pool tasks by initially writing to faster CS ===