View Issue Details

IDCategoryLast Update
0008265bugs2021-09-16 13:34
ReporterunfaAssigned To 
Reproducibilityalways 
Status feedbackResolutionopen 
Product Version6.0 
Fixed in Version 
Summary0008265: Referenced audio files seem to be silently cached somewhere and not updated on file change
DescriptionWhen mastering music I always import my mixes without copying them, so that if I re-export them form my individual sessions, they'll automatically update in the master session.

However in Ardour 6 this workflow seems to be broken.

Importing a file without copying it seems to work initially, but subsequent changes to the source files are somehow ignored, as if Ardour cached the file somewhere and refused to reload it.

Even if I delete the original region, and re-import the new version Ardour sill somehow uses the old version of the file.
I don't know how is this possible, other than system RAM cache, because Ardour is not storing a copy in interchange, or external.

This issue means every time I correct the mix, I will not only have to re-export it, but also manually delete it from the master session, clear unused regions, restart Ardour and hope that when I import new version in it'll actually use it, instead of some cached old one.

I would like Ardour to reference the external (non-copied) files and maybe update the waveforms if it detects a change in the file (file modification date and MD5 sum verification?).

Working with referenced files is a very useful tool, and I'm really sad to see it not working right now in Ardour 6.
I had some issues with it in Ardour 5 before, but right now it doesn't seem to work at all.
TagsNo tags attached.

Activities

alexmitchellmus

2020-07-07 14:30

reporter   ~0024627

I have noticed this issue as well. It is very important, especially if one is using modifying the audio files in another software.

paul

2020-07-07 17:31

administrator   ~0024628

" Even if I delete the original region, and re-import the new version Ardour sill somehow uses the old version of the file."

How are you determining this? Listening, or looking?

unfa

2020-07-07 18:45

reporter   ~0024629

By listening - I know that the waveforms are cached, so I would try to clear the waveform cache , but it doesn't update the source files.
I wonder if this could be caused by Linux disk read cache?

x42

2020-07-08 00:35

administrator   ~0024632

How are you embedding the file? Menu > Session > Import ? and disabling "Copy files to session"?
Can you check the .ardour session file that the session indeed references an external file with absolute path?

If you're using Drag/Drop. in Ardour6 you have to disable "Preferences > General > Session > Drag and drop import always copies files to session"

unfa

2020-07-08 07:31

reporter   ~0024634

I think I've been using both Import dialog with "Copy files to session" disabled, and simply dragging the files into the session timeline.

I haven't tested this yet in Ardour 6.

paul

2020-07-08 13:51

administrator   ~0024635

Drag-n-drop has no options associated with it, and does not follow the settings in the dialog (which you could argue is a bug or usability problem; you might be right although it's not 100% clear). Drag-n-drop will always copy the files into the session (at this time).

alexmitchellmus

2020-07-08 16:49

reporter   ~0024639

DnD should either:

* Follow the set preference
Or
* Present the user with a dialogue: copy or link. (For all the files associated with the DnD action)

paul

2020-07-08 17:03

administrator   ~0024641

There will never be a dialog for DnD. It's too cumbersome. If you want control over how import/embed happens, use Ctrl-i (the import dialog)

alexmitchellmus

2020-07-08 17:05

reporter   ~0024642

Ok, well I think DnD should follow the setting then at very least. As currently it's confusing.

alexmitchellmus

2020-07-08 17:07

reporter   ~0024643

Wait... Robin stated DnD follows the settings? Is that correct?

alexmitchellmus

2020-07-08 17:07

reporter   ~0024644

Wait... Robin stated DnD follows the settings? Is that correct?

paul

2020-07-08 17:15

administrator   ~0024646

Robin is correct and I am wrong. Nothing new there :(

paul

2020-07-08 17:16

administrator   ~0024647

Ah, no, we are both right :)

There is a global option found ONLY in the preferences editor. DnD follows that.

This is not the same as the control found in the import dialog.

alexmitchellmus

2020-07-08 17:22

reporter   ~0024648

I feel that it can still be a source of confusion.

Without a popup dialogue for DnD possibly DnD should only ever copy.

Linking is an advanced use case, as projects could easily be damaged if moved around and a user forgot what the DnD setting was.

paul

2020-07-08 17:55

administrator   ~0024649

"possibly DnD should only ever copy."

which is why there's an option called "only ever copy imported files". Because "possibly" isn't the same as "definitely"

alexmitchellmus

2020-07-08 18:02

reporter   ~0024650

Sorry, I mean:

Possibly (as in possibly it should be locked), DnD should definitely only ever copy.

alexmitchellmus

2020-07-08 18:04

reporter   ~0024651

Unless there was a dialogue. So it was clear to the user.

x42

2020-07-08 18:33

administrator   ~0024652

Last edited: 2020-07-08 18:34

View 2 revisions

When the preference is *disabled*, and embedding is allowed. Embedding/linking is the default drop action. This is indicated by a small link-icon.
Like in any file-browser, holding ctrl will copy the file instead (shown by a small plus icon).

Note that this is only relevant when importing audio (drag/drop of files from a file-browser or desktop). DnD from Ardour's source-list (editor sidebar, previously imported to recorded files) is unrelated.
Also note that some files must be imported. mp3s or files where the sample-rate mismatches. Those are converted or resampled.

The preference "Drag and drop import always copies files to session" was off by default in Ardour5, in Ardour6 it's on by default.

@unfa is this the issue? Was the file accidentally copied instead of linked, or is the issue deeper?
Could you attach the .ardour session file (or check yourself if the file was copied into the session, and is not referenced by absolute path in the .ardour file). Thanks.

alexmitchellmus

2020-07-08 18:45

reporter   ~0024653

Thanks Robin and Paul.

One question. Currently is there an indication of the media on the timeline that is linked/internal?

I think this would be less confusing if it was clear what the relationship the session had with its media.

@x42 you mentioned an icon? Is that on the timeline media?

kiilerix

2021-09-01 12:34

reporter   ~0026129

Using Ardour 6.8, I see the same problem as unfa seemed to report:
"Importing a file without copying it seems to work initially, but subsequent changes to the source files are somehow ignored, as if Ardour cached the file somewhere and refused to reload it."

It seems like the discussion digressed into defaults for "Copy files to session", but the original report was clear that it was "without copying", and "Ardour is not storing a copy in interchange, or external".

Could there be other reasons that changes to imported files don't show up? Are there some other caches or copies in memory?

What behaviour should we expect when an imported file is changed? Should the file change be detected immediately, or after a restart of Ardour?

paul

2021-09-01 14:11

administrator   ~0026130

It would not be detected immediately - Ardour does not run a file monitor on the files used in a session. But it should be noticed on restart/reload of the session.

alexmitchellmus

2021-09-01 16:26

reporter   ~0026131

Could a wait timer check the files referenced for new modified timestamp?
At say 1 second intervals?

kiilerix

2021-09-01 21:49

reporter   ~0026134

Paul, thanks for clarifying the expected behaviour. I think I already tried restart without success, but I will take a closer look.

Alex: FWIW, I don't like the idea of polling every second. The better and "perfect" solution would be to subscribe to file system events whenever one of the source files change. But that would still inherently be "undefined behaviour" if it happened during playback/export, so it is perhaps not worth it.

A couple of change ideas that would have worked smoothly for me:

1. It seems like it currently is a silent no-op to import a source file that already exist. A "sorry dave, I can't let you do that" error message would be slightly better. But the ideal for me would be if it re-imported the file.

2. "Rebuild Peak Files" seems to be a solution to a very similar problem/use case. I tried it, but it didn't solve my problem. It could perhaps be generalized to "Reload Source Files" - with the obvious side effect that it rebuilt these "peak files". (It seems like "peak files" is an optimization implementation detail that rather shouldn't be exposed in the UI.)

Issue History

Date Modified Username Field Change
2020-06-24 09:35 unfa New Issue
2020-07-07 14:30 alexmitchellmus Note Added: 0024627
2020-07-07 17:31 paul Note Added: 0024628
2020-07-07 18:45 unfa Note Added: 0024629
2020-07-08 00:35 x42 Note Added: 0024632
2020-07-08 07:31 unfa Note Added: 0024634
2020-07-08 13:51 paul Note Added: 0024635
2020-07-08 16:49 alexmitchellmus Note Added: 0024639
2020-07-08 17:03 paul Note Added: 0024641
2020-07-08 17:05 alexmitchellmus Note Added: 0024642
2020-07-08 17:07 alexmitchellmus Note Added: 0024643
2020-07-08 17:07 alexmitchellmus Note Added: 0024644
2020-07-08 17:15 paul Note Added: 0024646
2020-07-08 17:16 paul Note Added: 0024647
2020-07-08 17:22 alexmitchellmus Note Added: 0024648
2020-07-08 17:55 paul Note Added: 0024649
2020-07-08 18:02 alexmitchellmus Note Added: 0024650
2020-07-08 18:04 alexmitchellmus Note Added: 0024651
2020-07-08 18:33 x42 Note Added: 0024652
2020-07-08 18:34 x42 Assigned To => x42
2020-07-08 18:34 x42 Status new => feedback
2020-07-08 18:34 x42 Note Edited: 0024652 View Revisions
2020-07-08 18:45 alexmitchellmus Note Added: 0024653
2021-09-01 12:34 kiilerix Note Added: 0026129
2021-09-01 14:11 paul Note Added: 0026130
2021-09-01 15:53 x42 Assigned To x42 =>
2021-09-01 16:26 alexmitchellmus Note Added: 0026131
2021-09-01 21:49 kiilerix Note Added: 0026134