Feature request: Recording path fail over
I recently found out that my SS instance was balking on startup due to the NAS being cranky.
This was causing a fair bit of difficulty on the SS host.
It would be great to have a timeout and fail over to an alternate path, with alert, when this occurs.
This was causing a fair bit of difficulty on the SS host.
It would be great to have a timeout and fail over to an alternate path, with alert, when this occurs.
Comments
This state is occurring when I added nightly scheduled restarts of the SSmac.
yes I am perusing the issue with the drive ( it's a raid ) manufacturer.
part of the thread dump
================
Process: SecuritySpy [48852]
Path: /Applications/SecuritySpy.app/Contents/MacOS/SecuritySpy
Architecture: i386
Parent: launchd [290]
UID: 501
Task size: 2650 pages
CPU Time: 0.007s
Thread 0x198296 priority 46
51 start + 53 (SecuritySpy) [0x24f5]
51 main + 69 (SecuritySpy) [0x2575]
51 Initialise + 4210 (SecuritySpy) [0x36c2]
51 SetUpCaptureDestination + 282 (SecuritySpy) [0xde3a]
51 UpdateCaptureDestinationFromAliasBackup + 336 (SecuritySpy) [0x56810]
51 ResolveAliasWithMountFlags + 37 (CarbonCore) [0x9140c5d2]
51 _ResolveAliasWithMountFlags + 103 (CarbonCore) [0x9140c641]
51 FSResolveAliasWithMountFlagsInternal + 133 (CarbonCore) [0x9140b3c2]
51 FSMatchAliasInternal + 2228 (CarbonCore) [0x913d22ea]
51 FindTarget + 266 (CarbonCore) [0x913d385c]
51 _FSGetCatalogInfoByName + 282 (CarbonCore) [0x913f4919]
51 FSMount::getattrs(unsigned long, char const*, unsigned long, unsigned long, FSAttributeInfo*, unsigned long, unsigned char*) + 229 (CarbonCore) [0x9142378f]
51 FSMount::_getattrs(unsigned long, char const*, unsigned long, unsigned long, FSAttributeInfo*, unsigned long, unsigned char*) + 308 (CarbonCore) [0x914a9db8]
51 GetVolFSAttributes(FSMount*, unsigned long, char const*, unsigned long, unsigned long, FSAttributeInfo*, unsigned long, unsigned long, FSVolAttributeInfo*, unsigned char*) + 619 (CarbonCore) [0x91438d53]
51 __getattrlist + 10 (libsystem_kernel.dylib) [0x9027e1ae]
*51 ??? (mach_kernel + 849280) [0xffffff80002cf580]
*51 unix_syscall + 467 (mach_kernel) [0xffffff80005e9573]
*51 getattrlist + 144 (mach_kernel) [0xffffff80002e1240]
*51 namei + 1454 (mach_kernel) [0xffffff80002f25fe]
*51 lookup + 556 (mach_kernel) [0xffffff80002f2c1c]
*51 VNOP_LOOKUP + 52 (mach_kernel) [0xffffff8000318d04]
*51 hfs_vnop_lookup + 1144 (mach_kernel) [0xffffff8000501138]
*51 cat_lookup + 152 (mach_kernel) [0xffffff80004f1358]
*51 ??? (mach_kernel + 3085909) [0xffffff80004f1655]
*51 BTSearchRecord + 179 (mach_kernel) [0xffffff8000525293]
*51 GetNode + 44 (mach_kernel) [0xffffff80005282bc]
*51 GetBTreeBlock + 182 (mach_kernel) [0xffffff80004ef756]
*51 buf_biowait + 121 (mach_kernel) [0xffffff80002e4bc9]
*51 msleep + 116 (mach_kernel) [0xffffff80005722f4]
*51 ??? (mach_kernel + 3612422) [0xffffff8000571f06]
*51 lck_mtx_sleep + 78 (mach_kernel) [0xffffff800022670e]
*51 thread_block_reason + 332 (mach_kernel) [0xffffff800022dcac]
*51 ??? (mach_kernel + 190945) [0xffffff800022e9e1]
*51 machine_switch_context + 366 (mach_kernel) [0xffffff80002b4bce]
Have you tried adding the drive to the list of login items, so that it auto-mounts upon startup? See this guide: How to Automatically Connect to a Network Drive at Login in OS X.
However it failed on a firmware update :P so I'll only be doing this to complete this thread ... to help the Future!
( yes the volume is listed in the Login Items, as is SSpy )
But I am not getting any fail-over from SSpy 4.2.9. Infact SS locks up on start and requires force quits. ( I can append a dump again )
specs os x 10.12.6 / Mini 16G 2.6Ghz i7 -> QNAP TS-670 Pro 4.3.5.0760
on that machine I am also running the SS webserver, and am monitoring that from a kiosk-configured RPi / HP Slate 21 combo.
Soo anyway this was running fine a month or so after the QNAP upgrade, but started getting cranky 12-12-2018, with repeated logging of forced wakeups ( wakupes_resource.diag) leading to the current condition ( last 2 days ) of .hang logs.
from the casual user the forced wakeup looks like SSpy crashes and auto restarts. The crash alert remains on the screen, but SSpy is running.
In trying to work this out SSPY locks up restart even after manually mounting the NAS volumes - so it should be finding the proper place to store.
As a detail it locks up displaying 'Initialising...' and some times at whatever 'looking for storage location' dialogs.
So I have a couple of questions.
1) can I manually edit the SSpy preferences file in ~/Library/Preferences/Security Spy Preferences /vNN ( where NN in my case is == 75 ) Why, to change the volume points to local disk so I can boot SSpy to continue to debug. I might need a hex editor but I don't know if you're checksumming etc.
2) Can I use NFS mounts SSpy? I am considering hard-mounting NFS mounts for my /Volumes/Cameras and leaving AFP.
in the following entry
== start ( probably mangled in the Forum's display.
L\A���$A����/Volumes/Cameras LEFT BRACKET afp://baduhq@BaduStorage(AFP)._afpovertcp._tcp.local/Cameras Server H �=A���$D7EB1663-831F-3A78-A6D1-52D9689E341F�/T����� � \ � p � ��DP\P�4f63b6d18e8dbb9c14294c74700de50b0d4b5ab6;00000000;0000000000000020;com.apple.app-sandbox.read-write;8|�@� h � � �P �����00000000015;/volumes/cameras/securityspy������
=== end
I am considering changing the
LEFT BRACKET afp://baduhq@BaduStorage(AFP)._afpovertcp._tcp.local/Cameras
to
LEFT BRACKET smb://baduhq@BaduStorage._smb._tcp.local/Cameras
after doing a remount via SMB. My axis cameras are having no problem managing their rule-based image uploads via SMB to this NAS, so ... it seems more reliable than AFP.
anyhoo this would allow me to get past the blocking start.
( edits for characters )
Applying the Hex Fiend to the Properties file worked. I was able to restart SSpy. There was a quick dialog for accepting the license agreement.
I am still curious about NFS mounts. I am going to see ( in ust a moment I suppose ) how well SMB holds up.
When SecuritySpy looks for a capture destination that it can't find, it should give up after some time and continue loading. If you hold the escape key while it's looking, it should give up sooner (or immediately).
However, we have seen some cases of hangs within a system call that is used by SecuritySpy to find/mount network drives. We haven't found a good solution for this yet, because the problem isn't in our code. The workaround for this seems to be the following:
- Turn off all network interfaces via the Network system preference (i.e. turn off WiFi and unplug any Ethernet cable connected to your Mac).
- Load SecuritySpy. With no network connection, it shouldn't hang.
- Once SecuritySpy has opened, go to the Preferences window and reset all custom capture destinations to their default value on the system drive.
- Then quit SecuritySpy and reinstate the network.
You should now be able to open SecuritySpy and set up the custom capture destination again to point to the NAS. I would strongly recommend you use AFP to mount the NAS rather than SMB.
It booted well, and I was able to double check that switching to SMB worked fin. All masks, for example, were retained. all accounts for the Axis pool. etc.
perhaps I'm just good? :P anyway I am old and have been doing stuff for a long time
I have a ticket in with QNAP and I'll continue to update this thread.
(the machine is racked & headless so it's a bit of a 'scene' to get to it and attach gear. Which is why I didn't simply do that. I will use that technique as a later resort.
Would you like the hang & restart dumps from Console? no biggie to send )
AFAIK AFP is deprecated, however useful it is.
If you have retained your settings after manually editing the Preferences file that definitely means you have skill!
The issue with the protocol for network volumes is with SecuritySpy's auto-deletion feature. For this to work reliably, SecuritySpy needs to read certain information from the files such as their creation date and size, as well as the amount of free bytes remaining on the volume. Across all system versions that SecuritySpy currently supports (macOS 10.7 - 10.14), AFP Is the only one that works reliably for this.
However, if you are running a newer macOS version, it really shouldn't matter: AFP, NFS and SMB should all work fine. In the next major update to SecuritySpy we will have to drop support for older macOS system versions for a variety of reasons, so at that time we will update our advice about this.