Rig cards - remote rig monitoring and notifications by email and Telegram

[Ushu]

Hello everyone!

I've created a service for remote rig monitoring. First of all I made it for my own rigs but after time decide bring service to public because it's really useful

It called RIG cards and available here: Dowanlod Rig clinet app


Basic functions:
 

• Status of rigs with info for GPUs (temperatures, fan speeds, frequencies etc.) Data for each rig present on its own "card" (as I called it) where you can monitor params just looking to the website
• Hashrate monitoring
• Notifications to e-mail or Telegram messenger about different events like:
  - Rig don't respond / Rig online again
  - Temperatures of GPU out of range or back to range (limits can be set for each rig individually)
  - Lost GPU on rig
  - Rig was rebboted
• Mobile-friendly version of the website, so you can easily check your rigs from smartphone
• Rig rebooting on events like: overheating GPUs, low GPUs temperature, losing GPU on the rig
• Remore rig reboot
• Able to set personal temperature color ranges for rig

Soon:
 

• Hashrate monitoring Ready! notifications which corresponds to it (will be later)
• Summary card with short status for all rigs, total hashrate, calculated profit, possible troubles
• Actions like reboot on different events (for example, reboot when lost GPU) Done!
• Other impovements (you can see more detailed tasklist below)

Here is some screenshots of service main page:






My main goal was to make easy in use service which main function is rigs monitoring and not so monstrous as some others.
I'll be happy to get any feedback, suggestions etc.


So, how it works?

First you need to download client application that you then will launch on the rig system.
Client app is just Windows Powershell script, so it's opensource, you can check contents and be sure so there no any viruses or other bad things. Besides it uses library from well-known Open Hardware Monitor to get info from GPU sensors.

View of client window:


Next you can add new rig from web-interface or during first launch of client app. Choose rig name, set number of GPUs, and temp. limits if wish.
BTW if you already added rig you can download preconfigured client for needed rig and just launch on system.

For launch the client app start the start_client.bat file (shortcut for this file you can then add to start on windows boot)

adding rig from web


adding rig from client app

Every two minutes client app sends rig data to server. If in 2 min period not received any data from rig, it will marked with "yellow" status (like warning). After next 2-3 minutes if still no rig activity - rig will marked as Offline "red" status and you'll get notification about it through choosen channels in settings.

Also each time when server got data from rig we check given parameters and if something wrong (for example lost GPU or overheating) we also send notification to you (of course if you enabled it).
Temperature limits you can set globally for all rigs (Account - Your Profile - Global settings) or for each rig individually by clicking coghwheel icon for rig you need.
Moreover depends on temp. levels values will be colored, so just taking a look you can check is there everything good with your rigs. You can set you own temperature ranges for each color.
Here is also nice trick with low t° limit: in most cases if something happens with miner on rig (unexpected close, can't connect to pool) GPUs stop working and their temperatures going down. So if find the appropriate value you'll receive notification of low temp and it will mean that GPU don't work normal!

How different statuses looks like:



Here is notifications examples (e-mail and Telegram):






For using client app you need to have .NET Framework 4.5 and Powershell 5 on your rig systems (Win 10 have them by default, on Win7 usually need to install).

Client has auto update function, but in next releases I add setting to disable auto update if you wish.



Hashrate monitoring


Details about hashrate monitoring you can find in post:

Hashrate monitoring


Now you can see miner hashrate on the rig cards.

It looks like this:



not sure is this is the best displaying of hashrate monitoring, so if you have any suggestions or comments about it - tell.


Notifications that depends on hashrate values don't implemented yet. Just displaying. Notifications maybe will be added some later after full debugging of hashrate monitoring feature, and finally if such monitoring is demanded by users.


Attention! The client gets the hashrate value directly from the miner (and not from the pool or from anywhere another)!

So in the settings of the rig, you need to select the type of miner used, and the miner on the rig can be configured accordingly - activate api (this can usually be made in a .bat file, and for some miners the API is activated by default and you don’t need to do anything).

You can select a miner when adding a new rig or editing an existing. This feature is also available when editing groups of the rigs.
Tab - Hashrate.





When setting a miner, you can also specify the port on which its api is available, if the port is different from the default.
Make attention! It's NOT a pool port!

There you can also see the parameters that need to be added to the miner’s launch line in the bat-file for activating api.
Anyway if something is not clear - just ask me! 


At the moment, the following miners are supported:


List of supported miners:
 

• Claymore's Ethereum Miner
• PhoenixMiner Ethereum
• SRBMiner Cryptonight
• EWBF Equihash Miner
• Ethminer
• T-Rex
• TeamRedMiner
• NBMiner
• GMiner

Ready for suggestions for adding any other miners 
Tell me which miners you are using, which coins, pools and I'll try to add requested miners.


Info about hashrate value will be also displaying in the client app window (or if it is impossible to get data from the miner by some reason, will be displayed error message).




Remind that the client connects to the server every 100-110 seconds - so after changing any settings on the server, it takes some time for the client to receive them, and only after that monitoring will continue with the new rig configuration.

Of course for support hashrate monitoring, the client app has been updated - version is 0.7.3
(The update will occur automatically if you have not disabled the auto-update feature)



As usual, very waiting from you feedbacks about my service and hashrate monitoring functions in particular.
Tell how it works, maybe there are any difficulties or other.

List of supported miners:
 

• Claymore's Ethereum Miner
• PhoenixMiner Ethereum
• SRBMiner Cryptonight
• EWBF Equihash Miner
• Ethminer
• T-Rex
• TeamRedMiner
• NBMiner
• GMiner

Ask if you want to see miner that don't added yet 



How much?
It's totally free for now! No any limitations.
No any fees! And of course it is don't touch your miners.
It just collect info from hardware sensors to show you on web and send nortifications if any.



Known issues and bugs:
 

• It's beta now so I don't except appearing bugs, I'll try to fix asap.
• Notification about unexpected rig rebooting doesn't work (not implemented yet, thinking about better algorithm for it) done
• Not working with P102-100 or P106-100 — it depends on mode of NVIDIA drivers, solution you can find below in FAQ
• As I see, there no longer any issues here
  On first launch of start_client.bat on Windows 10 if you unpacked client.zip by Windows Explorer you'll get Windows SmartScreen warning about unknown app - it's normal. There are no warnings if you will unpack zip with Total Commander for example or any other filemanager. As I understood, windows generate such warning every time if you first time start newest .bat which was downloaded from somewhere. And after some time if there no problems, app will be added to their database. You'll get SmartScreen warning only once, on the next launch of app you don't get warning.

To Do list:
 

• Hashrate monitoring Done! and notifications (will be later)
• Notification for unexpected reboot event done
• Rig uptime and date/time of launch (for each rig on its card) done
• Sort & order of rigs' cards done
• Sort & order for GPUs on rig
• Summary card with short basic information about all rigs, total hashrate, existing problems and other
• Adding features to Telegram bot (like getting info for rig at request)
• Addition Languages (russian and others)
• Actions (like reboot) on different events, possibility to launch bat-file on event
• Tasks from server to rig (reboot, shutdown)
• Logo and favicon for website
• Dark/night theme (maybe even posibility to set your own background image)
• Integration with sonoff-tasmota devices through Mosquito servers (MQTT)

FAQ

How I can use Telegram bot?
To start using you just need to send your secret key to him. Key can be found at Account - Your profile - Profile - Secret key
At this time bot can only send notifications to you, but in future I wish to add some functions

Is it possible to send notification to few Telegram accounts at the same time?
First make group in Telegram with few people. Then if you open screen with bot info, at the menu you can choose Add to group. After adding to group send next service message to bot:
/secret your_secretkey_here
Now bot will send notifications to group!
.

How much I need to pay for using service?
It's totally free. But if service come popular I will need to charge for it (to pay for servers and other stuff). But even in this case it will very cheap, 1 USD per month per rig for example (I don't thought seriously about it yet)


GPU NVIDIA P102-100 or P106-100 not showing
It's depends on mode of NVIDIA drivers works. By default it work in TCC mode and because of it Open Hardware Monitor library that I use to get info from sensors can't get these parameters. Solution is simple - switch drivers to WDDM mode.


Attention! With some new version of drivers, the described method no longer works.
An alternative method of grabbing info for such cards has been introduced (in most cases, you will not be required to make any actionsl). Details here.



According to the official NVIDIA website:
Quote
To change the TCC mode, use the NVIDIA SMI utility. This is located by default at C:\Program Files\NVIDIA Corporation\NVSMI. Use the following syntax to change the TCC mode:
Code:
nvidia-smi -g {GPU_ID} -dm {0|1}
Code:
0 = WDDM
1 = TCC

So you need go to the C:\Program Files\NVIDIA Corporation\NVSMI directory and from command line run such command for every card pointed it "number" (GPU_ID)

Or (but I'm not sure 'cause I haven't such GPUs) to switch all GPUs on the rig to WDDM mode you can run
Code:
nvidia-smi -g -dm 0


Time on rig cards is wrong
By default time on new accounts in UTC timezone. You can set your timezone at Account - Your profile - Timezone

How often data refresh on the page with rig cards?
For now data refresh every 30 seconds (but remind, rigs send data every two minutes). If needed you can refresh it immediately with refresh icon on the navbar. After 20 minutes of idle you session will stopped.


How is coloring the GPU temperature values? Is it possible to change it?
This coloring is not relating with temperature limits that you set for rigs to get notifications. It just for visual usability. It is possible to set your own temperature ranges for each color.
Default temperature ranges is:
1-35 °C - blue
36-60 - green
61-70 - yellow
71-80 - orange
81 и вышe - red



How I can enable the log file?
Logging is disabled by default. If you need it you must turn on it manually.

To enable log, you need to put -Log 1 into start_client.bat batch file.
Or, it can also be activated by editing the config.json configuration file, adding the parameter "log": 1

link: RIG cards and available here: Dowanlod Rig clinet app

So, this thread is the only way to know all of the features of its services?
Checked on your website, it only showed login form and one only Nav Menu to register.

Good job, but may I'll try when next features added.

Soon:
 

• Hashrate monitoring (I'll plan to add Claymore, EWBF miners first) and notifications which corresponds to it
• Summary card with short status for all rigs, total hashrate, calculated profit, possible troubles
• Actions like reboot on different events (for example, reboot when lost GPU)
• Other impovements (you can see more detailed tasklist below)

Good Luck