Jump to content

TDS

From Resco's Wiki
Revision as of 12:34, 9 July 2024 by Jzambor (talk | contribs) (Created page with "{{Wikipedia|Tabular Data Stream}} {{WIP}} {{Beta}} According to Microsoft documentation, the [https://learn.microsoft.com/en-us/power-platform/admin/control-tds-settings Tabular Data Stream (TDS)] protocol is an application-level protocol used to transfer requests and responses between clients and database server systems. Resco Mobile CRM can use the TDS protocol, which allows for faster synchronization (compared to the [https://learn.microsoft.com/en-us/previous-v...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Wikipedia logo
Wikipedia has an article on a similar subject:
Warning Work in progress! We are in the process of updating the information on this page. Subject to change.
Warning This page describes a function that has not yet been publicly released, or has been released in beta / preview quality. Subject to change.

According to Microsoft documentation, the Tabular Data Stream (TDS) protocol is an application-level protocol used to transfer requests and responses between clients and database server systems.

Resco Mobile CRM can use the TDS protocol, which allows for faster synchronization (compared to the SOAP endpoint).

About TDS

  • The protocol is 4 decades old.
  • The TDS endpoint for Dataverse has been in preview since May 2020.
  • There may be breaking changes in the future.

About TDS-based sync

  • Introduced in release 17.1.
  • It can be used with Microsoft Dataverse (Power Platform, Dynamics 365) but not for on-premises installations.
  • Speeds up the record download phase of synchronization (on average, 3-10 times faster). Typically, downloading records is the longest phase of sync.
  • Applies to both full sync and incremental sync.
  • Does not support FetchXML options (if you have such options in the Sync Filter, sync falls back to classic endpoint)

Enabling TDS endpoint for your Power Platform environment

Enabling TDS on Resco side

You have several options how to enable TDS:

  • On project level: Edit an app project in Woodford, go to Configuration > Offline Data Sync > Sync Advanced, and enable Use TDS download method.
  • On table level: Woodford admins can also set up the behavior for each table in Woodford.
  • On user level: App users can enable this feature in the app's setup.

Why is TDS so fast

  • Whether you use TDS or FetchXML, SQL server's work is the same.
  • TDS is a binary protocol written at times when every byte mattered.
  • TDS uses TCP communication, which is faster than HTTP.
  • Fetch relies on SOAP protocol, which in turn uses HTTP protocol, which in turn uses TCP.
  • Fetch response formatting is complex and redundant and requires compression.

TDS in sync log

A new section in the sync log informs about TDS.

<Downloader CacheSize=1500MB UsedCache=1510MB PausedFor=70%/>
16:59:47 resco_question: 15998 recs, 12281ms, 767ms/1402K/page | Increase page (now 1000)
16:59:47 team: 124182 recs, 9937ms, 236ms/1010K/page
16:59:47 yy_country: 247 recs, 2859ms
...
<TdsReport>
(1) resco_question: TDS does not support fetch options
yy_country: 247 recs, 1160ms + MaxVer:561ms
(2) team: 124182 recs, 7767ms + MaxVer:405ms, pageUsed=57000, 0.066 ms/rec
...
(3) AVG(MaxVer) = 163ms
</TdsReport>
<Summary>
<Settings App='17.1.0.4' Threads='3' />
<Results Sent='0' Recv='18332902' Result='Aborted'/>
<Times Total='1866031ms' Prepare='15ms'/>
<FullSync Recv='18332902' TotalTime='1865875ms' Entitys='1865859ms'>
<SyncDownloader CacheSize='1500MB' UsedCache='1510MB' PausedFor='70%'/>
</FullSync>
<Analysis>
<DownloadPerformance>0.1 ms/rec (exceptional)</DownloadPerformance>
(4) <Tip><Tip002>SyncDownloader paused for 70%; consider cache size increase.</Tip002></Tip>
</Analysis>
(5) <ApiCalls Downloader='16' Total='16' />
</Summary>

Notes:

  1. The SyncFilter for the resco_question table contained the DisableRowGoal option. As TDS does not support FetchXML options, Resco Mobile CRM switched to standard Fetch communication.
  2. TDS uses dynamic paging. Page size is selected so that the result set takes at most 20 MB.
  3. TDS does not support SELECT MIN_ACTIVE_ROW_VERSION. We need an extra row fetch to get that info.
  4. Ignore this warning for TDS.
  5. TDS communication does not count against the API call limit.

Risks, caveats, bottlenecks

The download is too fast. Older devices may have trouble processing and saving the records to the offline database. The cache may become full, and the WAL log size may become large. Use modern devices with enough RAM and free disk space. Increase cache size.