Custom command improvement/discussion

DayCuts · 12-18-2014, 10:46 PM

First of all a potential bug/issue, unless I am misusing or missing something...

Take the following example...

Code:

%d[input]
{
command $replace(%f,"s","z",1) %1
}

Gives the following result...

Code:

[13:19:24] [L] command tezt testinput
[13:19:26] [L] 200- ... 
[13:19:27] [L] 200 Command Successful.
[13:19:24] [L] command anothertezt testinput
[13:19:26] [L] 200- ... 
[13:19:27] [L] 200 Command Successful.
[13:19:32] [L] testinput
[13:19:33] [L] 500 'testinput': Command not understood.

As you can see after the code blocks are performed on each item selection the raw string from %d[] is sent to the server. Not sure if this is intentional?

I realize in this basic example I could just replace the %1 with %d[] and remove the code block, and if i need to use the input multiple times within a code block i can reference %1 for the rest. Moving %d[] inside the code block will only prompt once (i assume intentionally) but will then send the raw string from %d[] to the server for each iteration.

I then tried doing things some other ways and came across some unexpected results...

Code:

command:
%d[input]
{
command $replace(%f,"s","z",1) $replace(%1,"e","o")
}
result:
[13:19:24] [L] command tezt
[13:19:26] [L] 200- ... 
[13:19:27] [L] 200 Command Successful.
[13:19:24] [L] command anothertezt
[13:19:26] [L] 200- ... 
[13:19:27] [L] 200 Command Successful.
[13:19:32] [L] testinput
[13:19:33] [L] 500 'testinput': Command not understood.

Code:

command:
command $replace(%f,"s","z",1) $replace(%d[input],"e","o")
result:
[13:19:24] [L] command tezt
[13:19:26] [L] 200- ... 
[13:19:27] [L] 200 Command Successful.
[13:19:24] [L] command anothertezt
[13:19:26] [L] 200- ... 
[13:19:27] [L] 200 Command Successful.

This indicates to me that multiples of the same token modifier can't be use in a single line of a custom command. I tried with $uppercase as the second modifier which worked.

In summery:
1. Should the string input from %d[], when placed at the start of the line, be sent to the server? And if so can this be solved by adding a /noop (no operation) command.
2. Is it intentional for a token modifier to be limited to one instance per line, or is this a bug that can be fixed? If it is intentional then a solution might be to add variable support to the custom command interface. /var <key> <value> and $var(<key>) (or %v[<key>] etc, whichever fits best with the syntax standard) with support for performing /var <key> $modifier(%v[<key>],...) (which might be a useful addition anyway)

bigstar · 12-19-2014, 06:37 PM

%d on a line by it self will send the result to the server, this is by design, you could do something like this

Code:

/echo %d[input]

However I have discovered an a logic flow problem with the order of commands when mixing commands inside and outside of a command block

Code:

TEST 1
{
TEST 2 - %f
TEST 3 - %f
}
TEST 4

The commands are executed in the wrong order.. as seen below
TEST 2 - zz
TEST 3 - zz
TEST 1
TEST 4

Testing with /echo prefix like this and the result is

Code:

/echo TEST 1
{
/echo TEST 2 - %f
/echo TEST 3 - %f
}
/echo TEST 4

TEST 1
TEST 2 - zz
TEST 3 - zz
TEST 4

The problem is with the way "/prefixed" commands vs direct commands are executed when the command contains command blocks.

And if you use multiple command blocks the blocks execute in reverse order. Its a bit of a mess and it will take me some time to hammer out these problems.

It appears that my token parser fails to check for multiple identical modifiers within the same command line, I will have to re-work this.

I could add variables that can be set and then replaced later but the format will need to be unique so that it doesn't conflict with existing token elements I think I'd need to do something like this /set @<name>@ <value> where the name is encased with @'s I can't use $, %, #, !

DayCuts · 12-23-2014, 07:58 AM

I did notice the ordering issue but did not realize it was so extensive and forgot to mention it.

I was unaware that /echo existed, it is not listed in the help file that I could see. Though /noop would be more appropriate since the point is to prevent it being sent outside of the parser but /echo certainly suffices, thanks.

Variables was an afterthought really but for the sake of discussion is there a particular reason why %key or $var(key) would be an issue? I guess it comes down to how you parser is designed. If it allows nested 'token modifiers' (a kind of misleading name for them) then $var(localvar) could be used with minimal effort without needing much change to the parser. If not then %localvar might seem a better solution (and would comply more closely with the %globalvar form already used) but would obviously need a little error checking to avoid conflicts with global vars (or blacklist them from being used with local vars). Time/date vars shouldn't interfere because they already use a different form.

In any case would a prefix or encapsulation be needed at all for /set? wouldn't the code to process that command know that the second token on the line is the key and 3-N is the value, without the need to be told by some non-alphanumeric indicator? Again I suppose this all depends on how the parser is designed.

bigstar · 12-23-2014, 11:07 AM

The command ordering issue doesn't appear to be as bad as I initially thought, though it is an unforeseen problem.
The order is modified when using <tokens> i.e. %f within the command block and should now be fixed.

At this time not all commands and settings are currently documented in the help file, some might be mentioned somewhere in the forums.

The /echo command was added to make it easier for me to test the result of regex and token modifier operations, you can also use /test with the same result.

There are also quite a few undocumented /debug commands that I have defined for testing and experimenting. such as /debug cache which you may have seen me mention on the forums.

We currently use /set <var> <value> for setting several internal variables (with more planned) though I don't recall they've been documented in the help file.

/set information-bar-active-color <value>
/set information-bar-normal-color <value>
/set toolbar-bg-color <value>
Where value can be entered as a HTML color code i.e. #FFFFFF or RGB color code 255 255 255

For example in the Site Manager > Selected Site > "Perform these commands after login" I use /set toolbar-bg-color #C4E25A to set the toolbar background color to help me identify the session connected to the flashfxp.com server, often I'm opening and/or closing many instances of FlashFXP for testing and this makes it easier for me to avoid closing the wrong window at a glance.

The reason we can't use %key is because the %<token> variables are a fixed length of 1 character, to use a variable length token identifier the parser needs a way to determine where the identifier ends.

My original thought to use @<name>@ as the variable name seems like the best solution at this point, I experimented with a few different ideas and so far this is the only solution that appears to work 100% and didn't require any major design refactoring.

You can give this test build a try
http://www.flashfxp.com/testr/5/ffxp5.0.0.3799-r5.zip

1. Fixes the command block order when the block contains tokens.
2. Adds support for "/set @<name>@ <value>" and then you can use @<name>@ from that point forward. The variable/value is preserved until you close FlashFXP. I will add an /unset command when I have more time.

Some examples:

Store the value of %d as @x@

Code:

/set @x@ %d
/echo @x@

Store the current path %p as @p@

Code:

/set @p@ %p
/echo @p@

I haven't spent as much time on this as I would have liked, with the holidays my schedule has been quite full.

DayCuts · 12-26-2014, 03:26 AM

Hope you have enjoying the holiday season!

Re: /set, again I was unaware of this but that is easily solved by using /var instead (which is often used in scripting language to represent local variables that are automatically unset after the current script is done). I see your point about % tokens being of restricted length in the current implementation. The way most do it is to break on any non-word character. A word character is usually defined as any letter, number, or underscore (and sometimes a dash is included). This way any percentage sign, space, comma, period, newline, etc would indicate the local variable name boundary without the need for @'s etc.

How hard such an implementation change might be of course again depends on the parser design. Use of PCRE for example would make variable replacement of the entire script quite simple (in comparison to say, a step through approach). But as you say this would most likely require significant design refactoring of your parser.

bigstar · 12-26-2014, 09:11 AM

Thanks.

When defining the variable I could omit the trailing @ but not but not for the inline use of it. There may be instances where the variable is used in a way that doesn't word break.

At this point I don't feel comfortable refactoring the script/command engine because I don't want to accidentally change a behavior of how something currently works. I have quite a few test cases however I know that it doesn't cover everything and it would be very easy for something to break and go unnoticed. such as the issue with the command execution order.

My main goal is to make it as usable as possible without limitations and while I understand that this syntax might overly complicate the logic it does seem to work in many conditions where any way else could possibly fail.

I found an issue with the new variable code, if you assign a command to a variable such as /select it doesn't perform the command after replacing the variable with the value.

i.e. this doesn't execute the /select command.

Code:

/set @A@ /select -f *.txt
@A@

DayCuts · 12-26-2014, 04:12 PM

I think it would be perfectly acceptable for /set @A@ to treat "/select -f *.txt" as a literal string and not re-evaluate it to allow your usage example at all. Commands by nature are expected to be at the beginning of the line for the most part (in this type of set up), and certainly that is ALWAYS the expectation for custom command scripting in flashfxp.

What I would generally expect is for "/set @A@ $something()" to evaluate $something() and store the resulting string literal as the value of @A@, meaning that NOT evaluating /select in that same manner would be inconsistent unless by design (that is, /cmd's must always be at the start of a line therefor a / will be treated as a string literal thereafter).

I can not really think of situations where you example would be used? All that comes to mind is the hash calculation commands, but then I am wondering how /crc32 "<filename>" is different to /echo $crc32("<filename>") or just $crc32("<filename>") on its own? and by extension is there actually a need for /crc32 etc. The only other design method I can think of at the moment would possibly be to not support your example of putting a /cmd in a @var@ and running it with just @var@... but rather set the result of said /cmd as the string literal value of @var@. In the latter I can at least come up with usage situations, though I doubt they would actually be utilized. That also keeps things at a decent level of consistency.

Again the suggestion of variables was an afterthought. Though they would be useful - it is clear they push against the boundaries of what one would expect from a basic command design interface and are a more complex thing to implement in the type of parser you have designed. Mostly continued this discussion because I happen to enjoy software 'engine' design and have written various low level 'engines' and parsers myself in the past.

bigstar · 12-27-2014, 09:02 AM

I've corrected the issue with the way @var@ is evaluated because with the above mentioned issue behavior was inconsistent when the command was preparsed. just like the out of order command issue when mixing raw commands with /commands that I previously fixed.

In this example @A@ was echoed to the console window as if it were executed but in reality it was just echoed and not actually performed, since this is very misleading I considered this behavior wrong.

Code:

/set @A@ /select -f *.txt
@A@

In this example the is performed as expected.

Code:

/set @A@ SITE HELP
@A@

The reason for the two different commands is that /crc32 displays the result directly in the status window using a predefined format.

Code:

[07:33:02] [L] [CMD] /crc32 flashfxp.log
[07:33:02] CRC32: a9ce9e68 File: F:\flashfxp.log (0.1 KB/s)

with $crc32(<filename>) it would return only the crc32 hash and this could be used to do things like create a sfv file.

Code:

# This will create a SFV file of all selected files
# using the standard SFV hash file format
# <filename> <crc32>
/writeln "%pverify.sfv" %f $crc32("%p%f")

I think your idea of allowing the user to set/get variables is a great idea and I wish I had thought of it

I will link to an updated build shortly with my latest changes.

DayCuts · 01-19-2015, 08:21 PM

Hi, couple of things that are all sort of related...

1. Just wondering what the current implementation status is for variables? and for example will "/set @A@ /select -f *.txt" set @A@ to the literal string "/select -f *.txt" or the result of it (based on previous post i am assuming literal string).

EDIT: Testing and confirmed for myself that my assumption is correct.

2. While attempting to write a new command i found myself unable to figure out a way to do something. However I am not sure if I am just forgetting about something, and I have not tried using variables yet hence the question above (if my assumption is correct then variables wont help)

What I am attempting to do it move files (on the remote side) but I am selecting the files to be moved with /select and can not think of a way to use /ren /enqueue (or raw commands) with the selected item(s). Can you think of a way to do this that I am not seeing?

3. While working on another command I noticed some unexpected behavior. Consider the following example.

/select ...
/delete selected
/transfer queue

If their are previously failed items already in the queue then /transfer queue resets them even through there are unfailed items also in the queue (the enqueue commands). Maybe the global transfer routine (that checks the queue and resets if needed) is not taking enqueued items into account?

4. Suggestion to update /delete (and maybe similar commands) to accept a [now] parameter at the end eg /delete selected now. This would perform the delete operation immediately. I understand this may not be as simple as it sounds.

5. Linked to both 2 and 3... a way to reference selected items similar to %f. This could then be used in other commands (like /ren). Again I understand this may not be as simple as it sounds. For such a token I would expect that the command, eg /ren, would be iterated for each item in the /select result that the token is referencing.

bigstar · 01-20-2015, 03:10 PM

1. Yes, as you discovered. Though the variables feature and how it works isn't 100% set in stone.

2. If you want to move all the items in the current directory to a specific folder you could do something like this

Code:

/selectall
/ren "%p%f" "/archived/%f"
/list

If that doesn't help, please give me a specific example of what you're trying to do.

3. This is the behavior when the delete operation command is executed. It will not run if any items in the queue are marked as failed, FlashFXP will reset and move these failed operations to the top of the queue before allowing the delete operation to run.. if the failed operation cannot be performed (fails over and over) then once the retry limit is reached the delete operation is also marked as failed.

4. The only reliable way to delete files is to do it via the queue (which can't be performed immediately), if you needed to delete a file immediately you could use "DELE %p%f"

5. Did you use pass absolute path name when using /ren? in many cases the absolute path is required, sometimes FlashFXP will try to guess and use the current dir but since the path may change due to a loss of connection or another command this is not reliable, especially if you're restoring a saved queue.

%f should work in most commands, I tested it with /ren and it worked as expected.
Also remember that most /macro's parameters must be double-quoted.

bigstar · 01-20-2015, 05:14 PM

/enqueue is a way for a script to include operations in the queue that would otherwise be performed immediately.

If you use for example the /ren command then the operation is performed immediately (not added to the queue), now depending on the situation this may not give the desired result.

Take a look at this code, since the /ren command is performed immediately the rename operation takes place before the files are transferred. FAILURE..

Code:

/selectall
/transfer selected
/ren "%p%f" "%p%f.downloaded"
/start

To get the desired result we need to enqueue the /ren command like this

Code:

/selectall
/transfer selected
/enqueue /ren "%p%f" "%p%f.downloaded"
/start

The reason I used /transfer selected above and not /queue selected is because the /queue selected command adds the items to the end of the queue where-as the /transfer selected command inserts the items in-line. This is somewhat confusing, I know

DayCuts · 01-26-2015, 04:46 AM

Re 3+4. Thank you, I see why it would be designed that way. I just happened to have items marked as failed when I was working on the command. I shall just remember to open a clean session to avoid this.

Re 2. Sorry I wasn't very clear about the selection process here... I am attempting to perform an action on a /select*ed set of items from within a sub folder. These items can not be known prior to entering the folder as the folder is itself entered using /cd %p%f. So %f is already populated. I tried using a sub command to break things up and test the internal handing of %f and got the following results...

Command TEST

Code:

/echo -A
{
/cd %p%f
/echo -B
/select -f rx: .*\.tmp$
/echo -C
/delete selected
/echo -D
[SUBTEST]
/echo -E
}
/echo -F
/uncd
/echo -G

Command SUBTEST

Code:

/echo -D1
/select -f rx: .*\.log$
/echo -D2
/enqueue /ren "%p%f" "%pLogs/%f"
/echo -D3

Manually select folder TESTING2 and perform command TEST.

Code:

[19:05:38] [L] [CMD] -A
[19:05:38] [L] [CMD] /cd /tools/bin/TESTING2
[19:05:38] [L] CWD /tools/bin/TESTING2
[19:05:38] [L] 250 CWD command successful.
[19:05:38] [L] PWD
[19:05:39] [L] 257 "/tools/bin/TESTING2" is current directory.
[19:05:39] [L] STAT -l
[19:05:40] [L] List Complete: 294 bytes in 0.88 seconds (0.3 KB/s)
[19:05:40] [L] [CMD] -B
[19:05:40] [L] [CMD] /select -f rx: .*\.tmp$
[19:05:40] [L] [CMD] -C
[19:05:40] [L] [CMD] /delete selected
#Correct item added to queue for deletion... "/tools/bin/TESTING2/rebuild.tmp"
[19:05:40] [L] [CMD] -D
[19:05:40] [L] [CMD] SUBTEST
[19:05:40] [L] [CMD] -E
[19:05:40] [L] [CMD] -F
[19:05:40] [L] [CMD] /uncd
[19:05:40] [L] CWD /tools/bin
[19:05:40] [L] 250 CWD command successful.
[19:05:40] [L] PWD
[19:05:40] [L] 257 "/tools/bin" is current directory.
[19:05:40] [L] STAT -l
[19:05:41] [L] List Complete: 7,618 bytes in 0.88 seconds (7.4 KB/s)
[19:05:41] [L] [CMD] -G
[19:05:41] [L] [CMD] -D1
[19:05:41] [L] [CMD] /select -f rx: .*\.log$
[19:05:41] [L] [CMD] -D2
#Unexpected item added to queue for rename... "/tools/bin/TESTING2/rebuild.tmp" to "/tools/bin/TESTING2/Logs/rebuild.tmp"
[19:05:41] [L] [CMD] -D3

Manually select folders TESTING and TESTING2 and perform command TEST.

Code:

[19:35:07] [L] [CMD] -A
[19:35:07] [L] [CMD] /cd /tools/bin/TESTING
[19:35:07] [L] CWD /tools/bin/TESTING
[19:35:07] [L] 250 CWD command successful.
[19:35:07] [L] PWD
[19:35:07] [L] 257 "/tools/bin/TESTING" is current directory.
[19:35:07] [L] STAT -l
[19:35:08] [L] List Complete: 294 bytes in 0.89 seconds (0.3 KB/s)
[19:35:08] [L] [CMD] -B
[19:35:08] [L] [CMD] /select -f rx: .*\.tmp$
[19:35:08] [L] [CMD] -C
[19:35:08] [L] [CMD] /delete selected
#Correct item added to queue for deletion... "/tools/bin/TESTING/rebuild.tmp"
[19:35:08] [L] [CMD] -D
[19:35:08] [L] [CMD] SUBTEST
[19:35:08] [L] [CMD] -E
[19:35:08] [L] [CMD] /cd /tools/bin/TESTING2
[19:35:08] [L] CWD /tools/bin/TESTING2
[19:35:09] [L] 250 CWD command successful.
[19:35:09] [L] PWD
[19:35:09] [L] 257 "/tools/bin/TESTING2" is current directory.
[19:35:09] [L] STAT -l
[19:35:10] [L] List Complete: 294 bytes in 0.87 seconds (0.3 KB/s)
[19:35:10] [L] [CMD] -B
[19:35:10] [L] [CMD] /select -f rx: .*\.tmp$
[19:35:10] [L] [CMD] -C
[19:35:10] [L] [CMD] /delete selected
#Correct item added to queue for deletion... "/tools/bin/TESTING2/rebuild.tmp"
[19:35:10] [L] [CMD] -D
[19:35:10] [L] [CMD] SUBTEST
[19:35:10] [L] [CMD] -E
[19:35:10] [L] [CMD] -F
[19:35:10] [L] [CMD] /uncd
[19:35:10] [L] CWD /tools/bin
[19:35:10] [L] 250 CWD command successful.
[19:35:10] [L] PWD
[19:35:11] [L] 257 "/tools/bin" is current directory.
[19:35:11] [L] STAT -l
[19:35:12] [L] List Complete: 7,618 bytes in 0.89 seconds (7.4 KB/s)
[19:35:12] [L] [CMD] -G
[19:35:12] [L] [CMD] -D1
[19:35:12] [L] [CMD] /select -f rx: .*\.log$
[19:35:12] [L] [CMD] -D2
#Unexpected item added to queue for rename... "/tools/bin/TESTING/rebuild.tmp" to "/tools/bin/TESTING/Logs/rebuild.tmp"
[19:35:12] [L] [CMD] -D3
[19:35:12] [L] [CMD] -D1
[19:35:12] [L] [CMD] /select -f rx: .*\.log$
[19:35:12] [L] [CMD] -D2
#Unexpected item added to queue for rename... "/tools/bin/TESTING2/rebuild.tmp" to "/tools/bin/TESTING2/Logs/rebuild.tmp"
[19:35:12] [L] [CMD] -D3

There are a few things to note in the results...
First of all I see that %f can in fact refer to a sub selection if outside of a command group already iterating %f. So %f is only static within its current command group {}. I had wondered about this hence the testing with a sub command.
Secondly the SUBTEST command is not performed until after everything in the TEST command, I am assuming this is not the intended behavior. As you can see from the second set of results it iterates over the command block for each manually selected %f when the command was launched, then continues to finish off the rest of TEST, then performs each iteration of the SUBTEST command.

I will continue to play around with the logic and ordering to see if I can get this to work, including maybe something like below, though I may just write it in C instead and load it as a raw command on the server. The latter is easy enough I just thought it would be more handy to have it with my client for better portability... and now that I have started the challenge to get it to work as expected has sucked me in lol.

Code:

{
[MARK ITEMS FOR DELE]
}
{
[SELECT MARKED ITEMS AND ENQUEUE /delete]
}
/markedclear
{
[MARK ITEMS FOR MOVE]
}
{
[SELECT MARKED ITEMS AND ENQUEUE /ren]
}
/markedclear

DayCuts · 01-26-2015, 05:34 AM

Upon further investigation it seems that the sub command in the above examples are in fact performed at the correct time but the echo's are not. Using the simplest examples i could to demonstrate...

Example 1 - raw commands are not performed at the expected times.

Code:

#CMD
A
{
/cd %p%f
[CMD2]
}
/uncd

#CMD2
B

#LOG
[21:01:27] [L] [CMD] /cd /tools/bin/TESTING
[21:01:27] [L] CWD /tools/bin/TESTING
[21:01:27] [L] 250 CWD command successful.
[21:01:27] [L] PWD
[21:01:28] [L] 257 "/tools/bin/TESTING" is current directory.
[21:01:28] [L] STAT -l
[21:01:29] [L] List Complete: 294 bytes in 0.94 seconds (0.3 KB/s)
[21:01:29] [L] [CMD] CMD2
[21:01:29] [L] [CMD] /cd /tools/bin/TESTING2
[21:01:29] [L] CWD /tools/bin/TESTING2
[21:01:29] [L] 250 CWD command successful.
[21:01:29] [L] PWD
[21:01:29] [L] 257 "/tools/bin/TESTING2" is current directory.
[21:01:29] [L] STAT -l
[21:01:30] [L] List Complete: 294 bytes in 0.93 seconds (0.3 KB/s)
[21:01:30] [L] [CMD] CMD2
[21:01:30] [L] A
[21:01:31] [L] 500 'A': Command not understood.
[21:01:31] [L] [CMD] /uncd
[21:01:31] [L] CWD /tools/bin
[21:01:31] [L] 250 CWD command successful.
[21:01:31] [L] PWD
[21:01:32] [L] 257 "/tools/bin" is current directory.
[21:01:32] [L] STAT -l
[21:01:32] [L] List Complete: 7,618 bytes in 0.95 seconds (7.4 KB/s)
[21:01:32] [L] B
[21:01:33] [L] 500 'B': Command not understood.
[21:01:33] [L] B
[21:01:33] [L] 500 'B': Command not understood.

Example 2 - /enqueue commands are performed at the expected times but sub command /echo's are not.

Code:

#CMD
/echo -- Enqueue A
/enqueue A
{
/cd %p%f
[CMD2]
}
/uncd

#CMD2
/echo -- Enqueue B
/enqueue B

#LOG
[21:11:57] [L] [CMD] -- Enqueue A
#A is correctly enqueued
[21:11:57] [L] [CMD] /cd /tools/bin/TESTING
[21:11:57] [L] CWD /tools/bin/TESTING
[21:11:57] [L] 250 CWD command successful.
[21:11:57] [L] PWD
[21:11:58] [L] 257 "/tools/bin/TESTING" is current directory.
[21:11:58] [L] STAT -l
[21:11:59] [L] List Complete: 294 bytes in 0.93 seconds (0.3 KB/s)
[21:11:59] [L] [CMD] CMD2
#B is correctly enqueued
[21:11:59] [L] [CMD] /cd /tools/bin/TESTING2
[21:11:59] [L] CWD /tools/bin/TESTING2
[21:11:59] [L] 250 CWD command successful.
[21:11:59] [L] PWD
[21:11:59] [L] 257 "/tools/bin/TESTING2" is current directory.
[21:11:59] [L] STAT -l
[21:12:00] [L] List Complete: 294 bytes in 0.93 seconds (0.3 KB/s)
[21:12:00] [L] [CMD] CMD2
#A is correctly enqueued again
[21:12:00] [L] [CMD] /uncd
[21:12:00] [L] CWD /tools/bin
[21:12:01] [L] 250 CWD command successful.
[21:12:01] [L] PWD
[21:12:01] [L] 257 "/tools/bin" is current directory.
[21:12:01] [L] STAT -l
[21:12:02] [L] List Complete: 7,618 bytes in 0.94 seconds (7.4 KB/s)
#These echos are misplaced.
[21:12:02] [L] [CMD] -- Enqueue B
[21:12:02] [L] [CMD] -- Enqueue B

Example 3 - /enqueue commands are performed at the expected times but sub command /echo's AND raw command are not

Code:

#CMD
/echo -- Enqueue A
/enqueue A
A
{
/cd %p%f
[CMD2]
}
/uncd

#CMD2
/echo -- Enqueue B
/enqueue B
B

#LOG
[21:20:39] [L] [CMD] -- Enqueue A
#A is correctly enqueued
[21:20:39] [L] [CMD] /cd /tools/bin/TESTING
[21:20:39] [L] CWD /tools/bin/TESTING
[21:20:39] [L] 250 CWD command successful.
[21:20:39] [L] PWD
[21:20:39] [L] 257 "/tools/bin/TESTING" is current directory.
[21:20:39] [L] STAT -l
[21:20:40] [L] List Complete: 294 bytes in 0.93 seconds (0.3 KB/s)
[21:20:40] [L] [CMD] CMD2
#B is correctly enqueued
[21:20:40] [L] [CMD] /cd /tools/bin/TESTING2
[21:20:40] [L] CWD /tools/bin/TESTING2
[21:20:41] [L] 250 CWD command successful.
[21:20:41] [L] PWD
[21:20:41] [L] 257 "/tools/bin/TESTING2" is current directory.
[21:20:41] [L] STAT -l
[21:20:42] [L] List Complete: 294 bytes in 0.94 seconds (0.3 KB/s)
[21:20:42] [L] [CMD] CMD2
#B is correctly enqueued again
#Misplaced raw command
[21:20:42] [L] A
[21:20:42] [L] 500 'A': Command not understood.
[21:20:42] [L] [CMD] /uncd
[21:20:42] [L] CWD /tools/bin
[21:20:43] [L] 250 CWD command successful.
[21:20:43] [L] PWD
[21:20:43] [L] 257 "/tools/bin" is current directory.
[21:20:43] [L] STAT -l
[21:20:44] [L] List Complete: 7,618 bytes in 0.94 seconds (7.4 KB/s)
#Misplaced echos and raw commands
[21:20:44] [L] [CMD] -- Enqueue B
[21:20:44] [L] B
[21:20:45] [L] 500 'B': Command not understood.
[21:20:45] [L] [CMD] -- Enqueue B
[21:20:45] [L] B
[21:20:45] [L] 500 'B': Command not understood.

As you can see order is not maintained correctly for certain things. What I would expect this to have evaluated to (internally) is...

Code:

/echo -- Enqueue A
/enqueue A
A
/cd /tools/bin/TESTING
/echo -- Enqueue B
/enqueue B
B
/cd /tools/bin/TESTING2
/echo -- Enqueue B
/enqueue B
B
/uncd

Instead it evaluated to

Code:

/echo -- Enqueue A
/enqueue A
/cd /tools/bin/TESTING
/enqueue B
/cd /tools/bin/TESTING2
/enqueue B
/uncd
A
/echo -- Enqueue B
B
/echo -- Enqueue B
B

DayCuts · 01-26-2015, 06:20 AM

Ok I figured out the mechanics behind the use of multiple %f lists and got my command working quite simply. %f simply must be set outside of the command group or sub command that iterates it. Logical enough... command groups or sub commands can be thought of as a 'foreach' in these circumstances. Not sure why it didn't occur to me before (as that is pretty standard for most programming languages) but for some reason I was thinking of %f in a static-global-variable nature and ignoring the obvious. When in fact %f is tied to its stack-level.

All the messing around did at least expose some oddities (above).

One other thing that I noticed after getting my command right and doing some error tolerance was that if there is a 550 error when trying to perform the remote rename the queue entry is still removed (rather than marked as failed). Is this intentional? I have not done a lot with enqueue commands so I am not entirely familiar with the expected behavior.

bigstar · 01-26-2015, 03:59 PM

Thank you for your feedback.

I attempted to reply to you several times today but each time I tried to write an example and show you some references I ran into some unexpected issues.

There are some problems with the order in which the scripts are being executed.

Ironically most of these problems seem to originate back to a single problem I tried to resolve awhile back where the order of the files in %f was reversed, it would appear that my fix and then additional fixes and changes have opened a huge can of worms.

I use a multi-stage partially recursive parsing and processing engine and it appears that somewhere along the lines I've overly complicated things because nothing works as expected.

I've spent most of the day trying to work things out but every time I fix one issue it introduces a new problem or changes the behavior in a way that may cause existing scripts to malfunction, though at this point I'm not entirely sure whats working correctly and what's not. I am pretty sure that I'll be rolling back all of my changes from today and starting fresh tomorrow.

To quickly answer some of your questions, yes when you use tokens such as %f in a command that command becomes a FOR-EACH LOOP or if the command is within a command block the entire command block becomes a FOR-EACH LOOP.

Chaining another command via [\sub\func] containing a command block can create its own FOR-EACH LOOP that is parsed within the sub context of the main loop but the the commands aren't actually executed until after the main FOR-EACH LOOP has been parsed. (If that makes any sense)

I have the following example that I was going to use to show you how things worked but at the moment it doesn't actually work as expected.

[WARNING - CODE DOESN'T WORK]

main function

Code:

/echo start
{
/cd %p%f
/selectall -f
[\sub\func1]
}
/clear cache %p
/cd %p
/echo end

sub function named "\sub\func1"

Code:

{
/move "%p%f" "../%f"
}
RMD %p

Below is a simplified output of the way the script should appear when run

Code:

[CMD] start
[CMD] /cd /base/sub1/
[CMD] /selectall -f
[CMD] \sub\func1
[CMD] /move "/base/sub1/sub1-file1.bin" "../sub1-file1.bin"
[CMD] /move "/base/sub1/sub1-file2.bin" "../sub1-file2.bin"
RMD /base/sub1/
[CMD] /cd /base/sub2
[CMD] /selectall -f
[CMD] \sub\func1
[CMD] /move "/base/sub1/sub2-file1.bin" "../sub2-file1.bin"
[CMD] /move "/base/sub1/sub2-file2.bin" "../sub2-file2.bin"
RMD /base/sub2/
[CMD] /clear cache /base/
[CMD] /cd /base/
[CMD] end