HOWTO: Data Logging

[Matt@pcmtec]

UPDATE: 12-09-22

Here is a link to the latest datalogging manual which can be viewed via help -> View Datalogger Manual

Introduction

The data logger is build into the PCMTEC editor, and allows you to seamlessly log your vehicle, and switch between the Nagivator and Flashing windows during a logging session, and then continue logging after reflashing and restarting the vehicle.

2.11 October 2022 Update

 

Once the Datalogger windows is opened, and the vehicle ignition turned on, the data logger will automatically connect to the Vehicle PCM:

Vehicles with a ZF automatic transmission can connect also the the transmission, using the 'Trans' button

External logging for recording AFRs, boost pressure or other user defined variables can be done using the DLP IO8-G Analogue/Digitial converter. This is covered in another HOWTO here

Selecting Channels

Use the Channels tab to select which Scalars you wish to view in the chart. Whilst the chart itself can display upto 16 channels (comrised of 4 charts, with 4 slots per chart), extra channels can be logged and saved and reviewed later on.

These can be selected live whilst connected to the PCM and TCM (ZF) and are then added to the logging session. 

Clicking on the Scalars tab displays the currently selected items. Anything greyed out is not displayed on the chart, but can be added using the right click menu

Allocating Chart Items

Right click and select a Chart item to allocated TQ_ETC_SRC to Chart 4 / Slot 1 for example

Otherwise use Multi Chart add to quickly add to the next available slot (if any are available)

That now displays in our chart

Using Scalar Delete will remove items from the chart, but they will still be recorded as part of the logging session

Using Chart Remove will free up the slot on the chart, but keep logging it and displayed in the Scalar section (greyed out)

Adjusting Ranges

There are several methods to adjust ranges of scalars from their defaults:

1. Right click Scalar and adjust range

2. Click and use the Zoom +/- buttons

3. Hover the axis and use the mouse scroll button, and adjust by hold and move up / down

4. Otherwise click the axis and use CTRL PAGE / CTRL DOWN keys

Use the PAGE UP / PAGE DOWN keys to adjust the position of the axis

5. Right click the axis and adjust the range using the window

All range adjustments are saved, and can be stored in the save Layout feature

Sorting Scalar items list

By default all items are sorted into their 'parent' groups, matching the Channel tree. If you Right click and disable 'auto sort' you can custom configure the ordering of the 'Scalars list'

Here I used CTRL UP and CTRL DOWN to move the scalar items with all temperature scalars positioned together

Enumerated Items

Some chart items like FUEL_SOURCE and TQ_ETC_SRC have meanings attached to the values, these are enumerated items. You can display the raw value by unticking the Enumerated tickbox

Update rates

Whilst there are no limitations on the number of channels you can log, on BA all channels will be 'polled'. Adding more channels will gradually reduce the update rate. On vehicles BF upwards, rapid packet reading is available for upto at least 15 DMRs and then polling available on top of this. 

If one of the channels you have selected requires a faster update rate, it can be forced to use rapid packet by selecting High Priority for that channel

The priority tickbox (left of the Scalar name) will be enabled for those High priority items you have selected. If none are selected then at least the 15 first DMRs in the list will be used for rapid packet.

Layout Load / Save

Use the Save / Load / Load append layout buttons (also available from the File menu) to keep a copy of your current chart configuration, including the items ticked, their chart slot and their ranges

Load append will add extra chart items to your current chart (ticked and their position if available) if they were not ticked previously. Note: Where a slot is already used, these will be ignored and not displayed

Layout files are saved in the PCMTec/Logging layouts folder, and saved as TLO files

Pause / Start / Clear buttons

During logging you can pause the live log updates (logging will still happen in the background) and then review your current log by moving the blue cursor

You can drag the cursor with the LHS mouse button and moving the mouse, or using the LEFT / RIGHT cursor keys. Holding the cursor keys will flip through the log page by page as it scrolls through the log. The legend/scalar values will update showing the current values at the cursor, even during a live logging session.

Clear button will clear out the current log. If the logging is over 10 seconds then you will be prompted if you want to keep the log. Logs are auto saved in case you wish to retrieve something later, then check out the auto save folder

Loading and Saving Logs

Use the Save icon to save your log as a TECLOG or CSV file (for loading into other software like MegaLogViewer for example)

Open the file will start in the PCMTec/Data logs folder, where your save files are located

CSV files from PCMTEC as well as other Ford tuning software export CSV format, are supported by the data logger for playback mode

In playback mode, you can press Start to begin playback, and pause to stop. 

Display Options

Smoothing: is on by default and will remove some of "stair casing" of the chart data

Auto range: is off by default. This will adjust the chart range based on what has been logged so far (or the range during playback).

Turning on will adjust the range to fit the current screen you see:

Auto scroll: will stop the chart scrolling if unticked

Legend: when unticked will hide the legend window if required

Adjusting units

Units must be adjusted whilst the PCM is disconnected. When doing this, the charts will be auto scaled to the new units type (for example from deg F to deg C)

Either adjust individually, from the Units dropdown:

Or adjust everything from bulk change:

 

Zooming

The time axis can be zoomed in and out using the scroll wheel on the mouse, when hovered over the main chart. Either zoom to the Left or Right hand side of the chart, upto a maximum of 1/8 second zoom level.

Using the keyboard, using UP and DOWN arrow keys will also perform similar zoom functions

Full short cut keys are available with F1 or the Shortcuts icon

Full short cut keys descriptions are in this HOWTO

 

[Romulus]

How can shift map be logged?

[Roland@pcmtec]

On 4/8/2020 at 7:51 AM, Romulus said:

How can shift map be logged?

I don't have a vehicle with a ZF to test with but there are several scalars in the ZF that look promising. Connect to the TCM and browse through the various shift map related scalars. Here are some ones that look interesting.

TID000568 - Actual Number of used shiftmap

TID002575- Current active shift (code)

TID000357- ID of used shift pattern

TID000351 - Actually used shift pattern

[Roland@pcmtec]

Video demonstration

 

[Roland@pcmtec]

For those of you who want to test this out at work here are some test files.

 

Example of 5 minutes driving.teclog

[Roland@pcmtec]

It was asked how to log torque. There is the actual torque value, the targetted torque (if undergoing torque reduction) and the 14 individual torque channels.

TQ_ACT -> Torque Actual
TQ_TARGET -> Targeted Torque (eg reduction)
TQ_LIM_x -> Torque Reduction Level (there are 14 of these that map to the torque requestors)

[fordsrule]

Wow

[Roland@pcmtec]

New public bug report forum for any other issues people find can be found here. Make sure your forum email address is the same as your registered email address if you post in here, otherwise send us an email with a link.

https://forum.pcmtec.com/forum/5-bug-reports/

[hulk]

Any one happy to share there layouts that thieve set 

up for logging.

trans,pressure,shift time,

engine , fuel, spark,boost,

keen to see what others have done

Thanks 

 

[AMR]

Where do I find the abbreviation to log rpm?

 

[Roland@pcmtec]

It's in the default.tlo layout file. Otherwise search for "engine speed" 

[AMR]

Thanks mate

[Roland@pcmtec]

Here are some early screenshots showing the Datalogger version 2. This has map tracing, histograms/scatterplot and a new revamped UI.

Here is an example showing a line trace over a spark multiplier map with a histogram showing min/max/avg spark values below. You can instantly create the histogram with axis pulled from a table by simply highlighting a section of the log and clicking add map trace.

Here is a screenshot of the new UI which will show min/max/avg on the left hand side of the selected area. There is also a new scroll bar which shows the rpm trace making it easy to find you WOT run after a dyno trace.

There will be more updates to come but this should be available around Christmas time. Thanks to @Matt for working very hard on this recently.

When it is complete and ready for public testing we will put a video up.

[fordsrule]

Hi, just wondering about Ver. 1.24, does it include map tracing?

I've seen reference to it but haven't been able to find it in the Datalogger or am I on the wrong track?

Thanks

[Roland@pcmtec]

The post above you says it's coming soon. It will be in the next major release of version 2.0

Aiming for a January release

[JETURBO]

Hey Roland & Team 

Great job on the scanner, What’s the chances ( unsure if it’s easy enough or not ) to have an option to have the colour/theme/layouts selectable to mimic HPT or SCT Advantage for people/workshops proficient in either of those to utilise a familiar display ? 
 

Not to say the 80’s colour selection isn’t good nostalgic but it’s crazy “Busy” to the eyes haha 

 

Probably more to the point can or has anyone worked their PCMtec layout to reflect as such ? 

[Roland@pcmtec]

You can change the theme via the theme menu. We are going to make the chart colours configurable at some stage in the future after map tracing and histograms are done. 

[JMSMotorsport]

Any update on the the Logger Stage 2?

[Roland@pcmtec]

Getting close. 

[Roland@pcmtec]

I'll put the reply here as well regarding poll rates.

The way the datalogger works is you get 15 rapid packet polls, these update at roughly 50ms. Then every thing else after this is polled as fast as it can, eg if you have 16 items you are logging, the 16th item will still log very fast. If you have 60 items, the 45 items after the first 15 will log quite slowly, roughly once a second.

We get a few 'freebies' that are broadcast data and always log fast irrespective of whether they are set to high priority. These are engine rpm, oil temp and a few others. This is something only we do, other companies don't take advantage of these.

If you are very curious about how it works you can right click on the name column, select column chooser, scroll down to 'rate' and add this column. This will show a number which is number of updates per second. For a fast dyno pull you need at least 250ms updates for rpm, boost, load, knock for it to be useful, otherwise the values will not line up and you will miss spikes of knock etc.

In future versions we will be setting high priority by default on knock, rpm, cam angle etc. Currently it has to be done manually if you have selected over 15 items.

[Loke]

any update on logger 2.0?

[Roland@pcmtec]

12 hours ago, Loke said:

any update on logger 2.0?

It is going through the final round of QA now, need to make sure it is right before it goes out.

[Roland@pcmtec]

Here is a link to the latest datalogging manual which can be viewed via help -> View Datalogger Manual

[rfh]

Hey Guys

Could someone please clarify when the spark source is "base" , which spark table is the pcm accessing? 

HAEDJK6

Thanks Rob

[Roland@pcmtec]

Base spark is MBT + any emissions or other adders. It will use the lowest value of all the different sources from MBT, BLK etc + the adders. Basically you can assume this is using MBT when it says base. Check auF2727 "Base spark retard for NVH", there are also a few emissions tables that subtract spark, these can come into play in some calibrations (Most have them zero'd). If these adders are active then it will likely report the source as base.

It can also report base spark when coming out of oscillation control, eg if you stab the throttle in first gear it will trim the spark to try and avoid driveline shuffle. It will then ramp back out of this mode. You might have seen tip in/tip out as the spark source prior to it saying base.

The intermediate value you can datalog for the various spark sources are.

MID90840 Spark Base

MID91036 Spark MBT

MID91051 Spark Base for NVH (noise vibration harmonics)

MID90837 Spark Base for Anti Fouling (zero'd out in all cals I've seen)

Also have a read of this if you haven't seen it before.

 

[rfh]

Much appreciated Roland

[Roland@pcmtec]

No worries. Good luck! 

[BeerTurbo]

We cant datalog unlicensed cars can we?

[Roland@pcmtec]

12 hours ago, BeerTurbo said:

We cant datalog unlicensed cars can we?

You can log unlicensed cars as long as they aren't tuner locked. 

[Stephe]

Howdy, I wonder if I could get some clarification regarding the saving and loading of the log file and corresponding layout? Reason is that I have tried many times to save a layout that I have setup for the log file I am reviewing. Every time I reload the log, the layout reverts to it's original configuration, and I have to set it up again. I have tried saving the layout, but when loading it back there is no data in the log tab, but I can go to the channels and check the required boxes and as I do that traces show in the log window, but I don't want that. Even loading the previous log means the layout has everything that I have unchecked back again, so I have to do it all over again, it just wastes time, so is there something I'm missing? I appreciate some help please.

[Roland@pcmtec]

Save a layout when you are live logging and see if that works as expected.

If not raise a support ticket with the steps youve taken.

[Stephe]

6 minutes ago, Roland@pcmtec said:

Save a layout when you are live logging and see if that works as expected.

If not raise a support ticket with the steps youve taken.

Thanks for replying Roland, I'll  give that a go

[Roland@pcmtec]

There are offline and online sessions. If you load a layout then load a previously saved log, it will use the layout from the old log. As your new layout may lack the parameters from the old layout so it doesn't make sense to load it in most cases. Layout are designed to be created and saved when online and logging a car, not when offline.

There is red text that says playback mode and online (I think) these are completely different sessions. Don't expect them to retain each other's sessions. Otherwise someone could send you a mustang log and it would wipe your falcon layout which would drive you mad.

[hjtrbo]

But why do we have to check the channel boxes when changing layouts offline? It's fucken annoying.

If I load a layout, just load it how I had it.

I can't change the dlp equations offline, that's annoying.

And has someone got confused with gauge pressure and absolute? Map channels are offset by baro when checking (wtf is) atmosphere. I.e. on the highway at 100km/h I'm 80kPa. If I select 'atmosphere' it reports 180kPa! It should be -20kPa.

With all due respect, I think you need to consider pausing the tri-core development and spend 3 weeks in a tune shop switching between HP, EFI and PCMTEC loggers.  

Ahh, rant over, if you need a field tester and advisor for the needed v3 or v4 logger development I've got my hand up.

[Stephe]

On 1/21/2024 at 5:41 PM, Roland@pcmtec said:

There are offline and online sessions. If you load a layout then load a previously saved log, it will use the layout from the old log. As your new layout may lack the parameters from the old layout so it doesn't make sense to load it in most cases. Layout are designed to be created and saved when online and logging a car, not when offline.

There is red text that says playback mode and online (I think) these are completely different sessions. Don't expect them to retain each other's sessions. Otherwise someone could send you a mustang log and it would wipe your falcon layout which would drive you mad.

Oh I see, the thing is, after a log of a car has been done, I like to review the log at my desk, as there is a lot of pids on screen I like to remove some so it's less confusing  and easier to read and understand, even moving channels in the log layout around makes it easier. So, if I come back to it the next day I have to go through all that again. So I can't save it afterall?

Thanks m8

I haven't got to trying your suggestion yet

[hjtrbo]

Great example of a terribly inefficient workflow.

 

[Roland@pcmtec]

It looks like the only issue is the checkboxes are being deselected when you re-load the layout. If you can raise a support ticket for this that may be a simple fix when people are free'd up to look at it. Once checked all of the items retain the order and chart position from the TLO file.

[Roland@pcmtec]

The best way to get any issues address is via support. Matt and Kirby no longer work for us, so most of your messages will not be read here or actioned.

We have a laundry list longer than anything you have raised of improvements. Now I will be extremely frank here, the problem is not that we are unaware of bugs/issues/inefficiencies, it is that we do not have any resources to resolve them.

The Falcon stopped being manufactured in volume in the late 2010s as such the Falcon product will no longer be updated beyond bug fixes as it is bordering on 20+ years old for many of the vehicles. If we were to continue developing/focusing on the Falcon, PCMTEC would cease to exist in a very short period of time. Simply look at companies like Nistune etc who focussed on one model of vehicle and did not move on, they all are now working part time or not at all.

The suggestion about going to a workshop and watching their workflow is exactly what we do. Every 6-12 months we send 3 people to a workshop for a day with a modified car, return it to stock, then film the tuner tuning the car recording everything they do, all issues, workflow, bugs etc. These visits provide a list of items that could never be completed in their entirety; however we can resolve the low hanging fruit and any major bugs. These visits are done soley for the Mustang/Tricore platform however in 2022 (the last datalogger update) we did do a Falcon visit.

For example the out of memory issues (this caused serious issues that could not be left unaddresed), we spent 12 months sourcing a new cable (OBDX) and an entirely new hardware version created (the OBDX FT with FEPS) as we requested it and spent many weeks back and forth resolving driver issues. We then spent a large amount of time creating a 64 bit editor/logger which was backwards compatible with x86 cables and machines. This resolved a huge number of issues with the Editor and Datalogger. Our statistics showed a huge 10 fold decrease in the number of crashes and shutdowns, bringing the number below 1%. The performance also increased dramatically.

Next major issue, categorisation, naming, units, equations. With over 200,000 parameters we had to spend 12 months building an internal tool to do this efficiently, not to mention finding staff who can actually use the tools and have time to do so.

The issues you bring up are real and acknowledged, however we simply do not have the resources to re-work the datalogger. The datalogger is a free product which cost in excess of $300,000 to develop. Software is incredibly time consuming, difficult and hard to build. We are a small team with a budget approximately 1% of HPTuners/SCT. They have 100s of developers, we have 3. They also have a 10 year head start on development (and revenue from a market 15x that of ours).

For PCMTEC to continue to grow and exist as a company we must innovate in a larger market, eg the Tricore platform that is Global, Australia and the Falcon is a tiny market and it does not bring in enough revenue to fund a team to build software at scale like other companies do. The goal which is now partially realised is to fund more developers to address issues you have raised and build out loss leader features (eg the datalogger which is free) along with innovating a platform which a greater number of users use and is still being built and developed by the OEMs. The reception of this product has been phenomenal and we have a order of magnitude of users who can benefit from our Tricore CustomOS.

Another other reason the datalogger is not at the top of the list is it is simply not used by our largest customers who do remote tuning via XCAL/NGauge/MyCal etc. They don't use the competitions software either, they use products like megalog viewer with CSVs and various other products. We must cater to those who are paying the bills and who will allow PCMTEC to grow and exist into the future.

We have spent 1000s of hours working with customers to improve their workflow, many of our larger customers can produce a tune in under 5 minutes, then do revisions in 10-15 minutes maximum. This includes building a multi tune slot tune, using parameter templates, reviewing datalogs and many other features. This is what drives what we work on.

For example the Tricore MultiTune wizard and parameter file system was wire-framed almost exclusively by Lund Racing, they received exclusive access for a period of time as a result and now everyone can benefit from it.

If you find a bug that is reproducible (opposed to a feature/workflow request) please submit it to support. Workflow requests are also appreciated, however they need to be something that the majority would use.

The video is great by the way, I just do not have anyone who I can assign it to at this stage. If it is an easy fix it will be addressed. We however absolutely do not want to break anything as a result of changing code that has been set in stone for close to 2 years now. So a 10 minute fix can become a 2-3 weeks R&D exercise due to QA and testing.

[Roland@pcmtec]

  

On 1/22/2024 at 6:00 AM, hjtrbo said:

And has someone got confused with gauge pressure and absolute? Map channels are offset by baro when checking (wtf is) atmosphere. I.e. on the highway at 100km/h I'm 80kPa. If I select 'atmosphere' it reports 180kPa! It should be -20kPa.

The Gauge units are actually correct by the way. My Mustang reads 450mmHG at idle on the factory dash. Ford incorrectly use their units internally for MAP in all their vehicles. Obviously there is not positive pressure in the manifold at idle, its a partial vacuum, however Ford don't believe in negative numbers for MAP, so it is what it is.

540mmHg is 10.4 psi, -4.25 psig

In the Falcon they use inHg as the default raw units for boost (IBP) and MAP. In their raw unadulterated form if you datalog these values you will see 10inHG or 5psi on the map sensor at idle, and 0 psi on the boost sensor (as its on the compressor outlet). Once on boost you will see 8psi (on a stock car) IBP and 22.7 psi on the MAP sensor.

As this doesn't really make sense (but it does to Ford and still does to this day) we have since fudged the raw units of the MAP sensor to pretend that the native units are InHga, this means you'll see a negative value on your map sensor when in vacuum. Beware however that every single table in the editor that references the map sensor does not have negative numbers, so whilst it may make more sense in the datalogger, you'll now have to add 14.7psi or 101.325 kpa to any MAP referenced table (or change your datalogger units to be psia/inHGa and compare them to psi/inHG. Boost is not affected as this has always read what you see on an oldschool analog boost gauge, it also doesn't read negative due to where it is positioned.

So we have just changed this so you will see -4psi at idle on the MAP sensor when selecting PSI, however if you need to reference this against a MAP referenced table (there aren't that may) you need to change the units to PSIA/inHGA. This might make you happier, but I guarantee it will confuse someone else who goes "why does my custom VE table start at 0 inHG/0kPa). You can't win here. If we get complaints or confusion about this we may change it back so that you will never see a negative MAP value. There isn't really a correct answer here and it is not possible to please everyone.

Here is how your datalog will look now if you take a new one today.

Just for fun, we left all the other DMRs alone in the Mustang so they match what you see on the dash and other dataloggers. MID57543 is is shared on the Mustang/Falcon. MID42519/MID02776 are not we are keeping these in the true ford spirit of "no negative numbers" with their raw native units as inHG (despite it not actually being inHG).

[hjtrbo]

I'm confused. Map is absolute referenced and boost is gauged referenced. We need those math channels

[Roland@pcmtec]

8 minutes ago, hjtrbo said:

I'm confused. Map is absolute referenced and boost is gauged referenced. We need those math channels

It is absolute referenced however its positive and all tables are positive. Look at the TMAP slope equation and the voltage input. The default units are inHG. However you only ever get a positive voltage, hence you only get positive numbers. The only way we can make it negative is by lying and saying that its inHga as the native raw units. Then when you select inHg or psi you'll see a negative number. It is not what Ford intended though.

 

[hjtrbo]

That's fine. I'm my simple little head boost = map - baro (0 floor). . Not sure if your earlier screen shot is in error with a 2ish psi gap. Perhaps I haven't put my earliest comment across correctly?

[Roland@pcmtec]

The raw assembly is:

map in inches of mercury = map slope(ayF0037) * map voltage + map offset  (auf0038)

eg map (inHG) = 15.983 * 0v + 0.4864 = 0.4864 inHg

Yeah it is absolute referenced, but Fords equation has a + not a subtract hence its a positive number.

Then for the boost sensor they use the opposite and you get a negative number (it caps at zero due to location). But Ford define both as inHG.

IBP (inHG) = 15.983 * 0v - 0.4864 = -0.4864 inHg

If Ford had done it correctly they would have set MAP as inHGa and IBP as inHG but they didn't.  Then to confuse everyone they use inHG for the desired boost table, but they also use inHG for all MAP referenced tables.

 

2 minutes ago, hjtrbo said:

That's fine. I'm my simple little head boost = map - baro (0 floor). . Not sure if your earlier screen shot is in error with a 2ish psi gap. Perhaps I haven't put my earliest comment across correctly?

2psi pressure drop across the intercooler.

 

 

[hjtrbo]

I think we're ok. 

Time for a bigger intercooler for you.

And math channels for us.

[hjtrbo]

To readdress my earlier point.. I think the selection drop down causes confusion. Should just be 'gauge' or 'absolute' text with the behind the scenes math resulting in what we expect to see. 

[hjtrbo]

@Roland@pcmtec I searched this site for obdx and x64. Apart from your post earlier today it came up empty. WRT the datalogger, what sort of improvements? In general is datalog throughput increased compared to the open port? If yes, how about zf logging throughput? Any back to back tests to give an idea of throughput rates?

Reason I ask, I've been enlightened to a new to me method of using ratio for shift tuning, it's the bomb. But its an offline process using MegaLogHD due to lack of live math channels. To work effectively it requires super high speed i.e. logging literally 3 channels (rpm, iss, oss) but as yet the data is still not coming in fast enough. I understand this might be a zf limitation. 

[Roland@pcmtec]

OBDX FT runs 64 bit mode. This generally makes the editor and datalogger run much faster and far more stable. It will log for 8 hours straight, if you don't like restarting software and leave the editor open for a week it will continue to run.

There have been no updates to the datalogger since this original thread was made (possibly a few bug fixes made there way in). Any improvements are side effects due to improvements in other parts of the software.

We have however continued to name and categorise DMRs, the actual back end software is unchanged though.

If you are tuning Falcons only I recommend using 2.xx as it includes a reduced template set (less memory and generally faster to connect and load).

3.xx. Is the same code base just with larger template and global name/description sets. This obviously adds overhead and will result in slightly reduced performance. It also has a mustang related features which are not enabled in 2.xx to lower the risk of new Mustang features breaking anything Falcon related.

[Roland@pcmtec]

22 hours ago, hjtrbo said:

I think we're ok. 

Time for a bigger intercooler for you.

And math channels for us.

We are considering adding some hard coded math channels for MAF logging (LTFT + STFT) / 2 in the interim, then potentially adding a UI for proper math channels, however this is likely going to be many months away at this stage.

[hjtrbo]

Perfecto. 

[hjtrbo]

ooh, not sure if this can be added to the math channel scope of works? DLP channels to be included as well? Use case is delta fuel pressure.

[DLP_IO_n(V)] x slope - offset + [dmr_baro(psi)] - [dmr_map(psia)]

If all is going well that number should be a constant 58 psi regardless of boost or vacuum. Makes identifying fuel pressure issues a sinch.