I was looking for a cache for messages to and from Streambase which could then be picked up from any number of excel addins
Redis seemed a good candidate for the key value cache - it's got reasonable traction in the community and performance.
The idea of the client is that it would pick up the initial value from Redis's get and subscribe to a channel with the same name for updates, similarly on publishing a value it would be set in the keyvalue pair and published in the same named channel.
I'm using the Excel add in in Delphi, which should be the easiest way of writing a native code add-in when one uses the AddIn Express libraries to avoid having to handle COM idiosyncracies (and the Addin Express support are really helpful)
Microsoft says on modest hardware one should be able to update 20,000 topics 3 times a second
http://code.google.com/p/delphi-redis/
https://github.com/ik5/redis_client.fpc
Kenny Kerr has written well and extensively on RTD addins
Alex Zhang has a nice overview and Microsoft support has the odd article worth reading
Essentially a series of strings in Excel in the RTD formula constitutes a topic that the RTD server must supply a value for.
The interchange between Excel and the RTD server can be grouped into four areas with corresponding methods in the RTDServer interface
ServerStart and ServerTerminate
starting and stopping the RTD server
ConnectData and DisconnectData
notifying the RTD server of new desired topics or ones no longer wanted
Heartbeat called by Excel to check the RTDServer is alive (1 being yes it is)
RefreshData, UpdateNotify
The RTD server lets Excel know that there is updated data via UpdateNotify. It doesn't say what data is updated.
Every throttleInterval seconds (set as Application.RTD.ThrottleInterval) in Excel, Excel calls RefreshData in the RTDServer and expects two pieces of information - the number of topics updated, and a 2 dimensional Variant array where each row is and array of the topicID and the new value for each updated value.
Note the TopicID is an integer assigned to the list of strings by Excel and given in the ConnectData call by Excel.
As the UpdateNotify doesn't say what is updated, there is no point in calling it more than once between each throttleInterval; one reasonable approach is just to stick the call on a timer and check then if there any updates (it needs to be called from the same threading appartment as the RTDServer is created in.
having RefreshData (which is called by the main Excel thread) do as little as possible is important for performance, and also on opening a spreadsheet with large numbers of topics, connectData should do as little as possible (maybe pushed the topics on a thread safe queue to be handled by the threads working with Redis)
Something else to be careful of is the handling of threads the updateNotify must be called from the Com appartment which typically means the thread that called the constructor for the RTDServer.
