BoxLang BIF -> Member Methods

Ok, so I’m ready to pull the trigger on one of our many departures from CF in BoxLang and I want to solicit a bit of feedback before proceeding. One of our driving goals of the 2-parser design and transpiler was being able to “fix” the warts of CF that we wish had been done differently. (I’m not throwing CF under the bus here-- BIFs long predated member methods, but if we were to write it again from scratch today, there are different decisions we would make!)

So the idea here is that any BIF (Built In Function) that operates on a data type would just be a member method if we were to design a language from scratch. This is how most other languages work and is pretty familiar outside of CF. I’ll also note, BoxLang doesn’t have any of the issues that plagued CF and Lucee for years where member methods wouldn’t always work on all data. Like how len( true ) was valid, but true.len() would error. CF/Lucee have mostly fixed those issues, and BL never had them :slight_smile: If a value can be cast to the type, then it will work!

So, what does this mean? It means BIFs like

arrayAppend( arr, value )
structKeyExists( str, key )
queryEach( qry, ()->{} )
uCase( name )

go away. They no longer exist. Gone from the language. Goodbye!
And instead, these now exist only in member form! (Just like most other language in existence)

arr.append( value )
str.keyExists( key )
qry.each( ()->{} )
name.uCase()

Nice, tidy, and object-oriented around the native types of our language. We’d still have lots of BIFs-- heck CF has nearly 900 BIFs! This just applies to BIFs that operate on a type and have a matching member method. Also note, CF WOULD REMAIN UNCHANGED AND WILL STILL HAVE ALL THE BIFS! Thanks to the magic of our separate CF parser and transpiler, we’d still parse CF code like always, but behind the scenes, we’d just transpile any of these BIFs over to use the member method syntax. And tools like our CF → BL CLI Transpiler would automatically refactor your code for you. So you really wouldn’t have to lift a finger to make this change other than remember to type the member version of each function when writing BL code from scratch in the future. Something the coding hints in our VSCode extension can help with.

Ok, so what BIFs ARE we talking about? We’ll, I built this list automatically from the metadata inside of BoxLang by pulling every BIF (and BIF alias) that also have matching member methods. I excluded the following BIFs because even though they have member versions, they aren’t a one-to-one mapping and don’t quite make sense:

  • hash()
  • fileInfo()
  • dump()
  • createODBCDateTime()

What’s left is 280 :exploding_head: BIFs we’d be effectively removing from BoxLang source code as reserved headless function names. There may be a couple you don’t recognize, such as getTime() which is new to BoxLang. You can find info for it in our docs.


So the questions are

  • Does this all make sense and like a good direction?
  • Have I missed any?
  • What perhaps DOESN’T belong in this list? Some of the number BIFs, for example, don’t have as strong of a precedent. The exp( myNum ) BIF for instance-- in java, you call Math.exp( myNum ) as a static method-- it’s NOT a member method on the actual number like myNum.exp(). So do ALL of these make sense, or should we keep some of these around as BIF or look at concepts like a static Math class for some of them? Or BIFs like JSONDeserialize() which I never really loved the member form of someString.JSONDeserailize(). That one still feels like perhaps it should remain a headless utility function.
  • Have we forgotten any compat issues this may cause?

Here is the full list for you to review. The right two columns are more of an implementation detail-- they track the argument position that the data type gets passed into the BIF when calling as a member method. Usually the first arg, but not always.

BIF Name Member Name Argument Name Argument Position
arrayappend() array.append() array 1
arrayavg() array.avg() array 1
arrayclear() array.clear() array 1
arraydelete() array.delete() array 1
arraydeletenocase() array.deletenocase() array 1
arraydeleteat() array.deleteat() array 1
arrayeach() array.each() array 1
arrayevery() array.every() array 1
arrayfilter() array.filter() array 1
arrayfind() array.find() array 1
arrayfindnocase() array.findnocase() array 1
arraycontains() array.contains() array 1
arraycontainsnocase() array.containsnocase() array 1
arrayfindall() array.findall() array 1
arrayfindallnocase() array.findallnocase() array 1
arrayfirst() array.first() array 1
arraygetmetadata() array.getmetadata() array 1
arrayindexexists() array.indexexists() array 1
arrayisdefined() array.isdefined() array 1
arrayinsertat() array.insertat() array 1
arraylast() array.last() array 1
arraymap() array.map() array 1
arraymax() array.max() array 1
arraymedian() array.median() array 1
arraymerge() array.merge() array1 1
arraymin() array.min() array 1
arraypop() array.pop() array 1
arrayprepend() array.prepend() array 1
arraypush() array.push() array 1
arrayrange() array.range() from 1
arrayreduce() array.reduce() array 1
arrayreduceright() array.reduceright() array 1
arrayresize() array.resize() array 1
arrayreverse() array.reverse() array 1
arrayset() array.set() array 1
arrayshift() array.shift() array 1
arrayslice() array.slice() array 1
arraymid() array.mid() array 1
arraysome() array.some() array 1
arraysort() array.sort() array 1
arraysplice() array.splice() array 1
arraysum() array.sum() array 1
arrayswap() array.swap() array 1
arraytolist() array.tolist() array 1
arraytostruct() array.tostruct() array 1
arrayunshift() array.unshift() array 1
jsondeserialize() string.jsondeserialize() json 1
jsonprettify() string.jsonprettify() var 1
jsonserialize() custom.tojson() var 1
tobase64() string.tobase64() string_or_object 1
tobinary() string.tobinary() base64_or_object 1
toimmutable() array.toimmutable() value 1
tomutable() array.tomutable() value 1
tostring() xml.tostring() value 1
isempty() string.isempty() value 1
isempty() query.isempty() value 1
structisempty() struct.isempty() value 1
arrayisempty() array.isempty() array 1
hmac() string.hmac() input 1
booleanformat() numeric.booleanformat() value 1
decimalformat() numeric.decimalformat() number 1
numberformat() numeric.numberformat() number 1
lsnumberformat() numeric.lsnumberformat() number 1
currencyformat() numeric.currencyformat() number 1
lscurrencyformat() numeric.lscurrencyformat() number 1
filereadline() file.readline() file 1
fileseek() file.seek() file 1
fileskipbytes() file.skipbytes() file 1
filesetaccessmode() file.setaccessmode() file 1
filesetattribute() file.setattribute() file 1
filesetlastmodified() file.setlastmodified() file 1
filewriteline() file.writeline() file 1
gettoken() string.gettoken() string 1
listappend() string.listappend() list 1
listavg() string.listavg() list 1
listchangedelims() string.listchangedelims() list 1
listcompact() string.listcompact() list 1
listtrim() string.listtrim() list 1
listdeleteat() string.listdeleteat() list 1
listeach() string.listeach() list 1
listevery() string.listevery() list 1
listfilter() string.listfilter() list 1
listfind() string.listfind() list 1
listfindnocase() string.listfindnocase() list 1
listcontains() string.listcontains() list 1
listcontainsnocase() string.listcontainsnocase() list 1
listgetat() string.listgetat() list 1
listfirst() string.listfirst() list 1
listlast() string.listlast() list 1
listindexexists() string.listindexexists() list 1
listinsertat() string.listinsertat() list 1
listitemtrim() string.listitemtrim() list 1
listlen() string.listlen() list 1
listmap() string.listmap() list 1
listprepend() string.listprepend() list 1
listqualify() string.listqualify() list 1
listreduceright() string.listreduceright() list 1
listremoveduplicates() string.listremoveduplicates() list 1
listrest() string.listrest() list 1
listsetat() string.listsetat() list 1
listsome() string.listsome() list 1
listsort() string.listsort() list 1
listtoarray() string.listtoarray() list 1
listvaluecount() string.listvaluecount() list 1
listvaluecountnocase() string.listvaluecountnocase() list 1
abs() numeric.abs() value 1
acos() numeric.acos() number 1
asin() numeric.asin() number 1
atn() numeric.atn() number 1
ceiling() numeric.ceiling() number 1
cos() numeric.cos() number 1
decrementvalue() numeric.decrementvalue() number 1
exp() numeric.exp() number 1
fix() numeric.fix() number 1
floor() numeric.floor() number 1
formatbasen() numeric.formatbasen() number 1
incrementvalue() numeric.incrementvalue() number 1
inputbasen() string.inputbasen() string 1
int() numeric.int() number 1
log() numeric.log() number 1
log10() numeric.log10() number 1
round() numeric.round() number 1
sgn() numeric.sgn() number 1
sin() numeric.sin() number 1
sqr() numeric.sqr() value 1
tan() numeric.tan() number 1
queryaddcolumn() query.addcolumn() query 1
queryaddrow() query.addrow() query 1
queryappend() query.append() query1 1
queryclear() query.clear() query 1
querycolumnarray() query.columnarray() query 1
querycolumncount() query.columncount() query 1
querycolumndata() query.columndata() query 1
querycolumnexists() query.columnexists() query 1
querycurrentrow() query.currentrow() query 1
querydeletecolumn() query.deletecolumn() query 1
querydeleterow() query.deleterow() query 1
queryeach() query.each() query 1
queryevery() query.every() query 1
queryfilter() query.filter() query 1
querygetcell() query.getcell() query 1
querygetresult() query.getresult() query 1
queryinsertat() query.insertat() query 1
querykeyexists() query.keyexists() query 1
querymap() query.map() query 1
queryprepend() query.prepend() query1 1
queryrecordcount() query.recordcount() query 1
queryreduce() query.reduce() query 1
queryreverse() query.reverse() query 1
queryrowdata() query.rowdata() query 1
queryrowswap() query.rowswap() query 1
querysetcell() query.setcell() query 1
querysetrow() query.setrow() query 1
queryslice() query.slice() query 1
querysome() query.some() query 1
querysort() query.sort() query 1
ascii() string.ascii() string 1
camelcase() string.camelcase() string 1
charsetdecode() string.charsetdecode() encoded_binary 1
compare() string.compare() string1 1
comparenocase() string.comparenocase() string1 1
find() string.find() string 2
findnocase() string.findnocase() string 2
findoneof() string.findoneof() string 2
insert() string.insert() string 2
ljustify() string.ljustify() string 1
rjustify() string.rjustify() string 1
kebabcase() string.kebabcase() string 1
lcase() string.lcase() string 1
left() string.left() string 1
listreduce() string.listreduce() list 1
ltrim() string.ltrim() string 1
mid() string.mid() string 1
paragraphformat() string.paragraphformat() string 1
pascalcase() string.pascalcase() string 1
refind() string.refind() string 2
refindnocase() string.refindnocase() string 2
rematch() string.rematch() string 2
rematchnocase() string.rematchnocase() string 2
removechars() string.removechars() string 1
replace() string.replace() string 1
replacelist() string.replacelist() string 1
replacelistnocase() string.replacelistnocase() string 1
replacenocase() string.replacenocase() string 1
rereplace() string.rereplace() string 1
rereplacenocase() string.rereplacenocase() string 1
reverse() string.reverse() string 1
right() string.right() string 1
rtrim() string.rtrim() string 1
slugify() string.slugify() string 1
snakecase() string.snakecase() string 1
spanexcluding() string.spanexcluding() string 1
spanincluding() string.spanincluding() string 1
sqlprettify() string.sqlprettify() sql 1
stringbind() string.bind() string 1
stringreduceright() string.stringreduceright() list 1
stringsome() string.stringsome() list 1
stringsort() string.stringsort() list 1
stripcr() string.stripcr() string 1
trim() string.trim() string 1
truefalseformat() string.truefalseformat() value 1
ucase() string.ucase() string 1
ucfirst() string.ucfirst() string 1
val() string.val() string 1
wrap() string.wrap() string 1
yesnoformat() string.yesnoformat() value 1
structappend() struct.append() struct1 1
structclear() struct.clear() structure 1
structcopy() struct.copy() struct 1
structdelete() struct.delete() struct 1
structeach() struct.each() struct 1
structequals() struct.equals() struct1 1
structevery() struct.every() struct 1
structfilter() struct.filter() struct 1
structfind() struct.find() struct 1
structfindkey() struct.findkey() struct 1
structfindvalue() struct.findvalue() struct 1
structget() struct.getfrompath() object 2
structgetmetadata() struct.getmetadata() struct 1
structinsert() struct.insert() struct 1
structiscasesensitive() struct.iscasesensitive() struct 1
structisordered() struct.isordered() struct 1
structkeyarray() struct.keyarray() structure 1
structkeyexists() struct.keyexists() struct 1
structkeylist() struct.keylist() structure 1
structkeytranslate() struct.keytranslate() struct 1
structmap() struct.map() struct 1
structreduce() struct.reduce() struct 1
structsome() struct.some() struct 1
structsort() struct.sort() struct 1
structtoquerystring() struct.toquerystring() struct 1
structtosorted() struct.tosorted() struct 1
structupdate() struct.update() struct 1
structvaluearray() struct.valuearray() struct 1
urlencodedformat() string.urlencodedformat() string 1
dateadd() datetime.add() date 3
datecompare() datetime.compare() date1 1
datediff() datetime.diff() date1 2
datetimeformat() datetime.format() date 1
dateformat() datetime.dateformat() date 1
timeformat() datetime.timeformat() date 1
lsdatetimeformat() datetime.lsdatetimeformat() date 1
lsdateformat() datetime.lsdateformat() date 1
lstimeformat() datetime.lstimeformat() date 1
lsparsedatetime() string.lsparsedatetime() date 1
lsweek() datetime.lsweek() date 1
lsdayofweek() datetime.lsdayofweek() date 1
parsedatetime() string.parsedatetime() date 1
year() datetime.year() date 1
quarter() datetime.quarter() date 1
month() datetime.month() date 1
monthasstring() datetime.monthasstring() date 1
monthshortasstring() datetime.monthshortasstring() date 1
day() datetime.day() date 1
dayofweek() datetime.dayofweek() date 1
dayofweekasstring() datetime.dayofweekasstring() date 1
dayofweekshortasstring() datetime.dayofweekshortasstring() date 1
daysinmonth() datetime.daysinmonth() date 1
daysinyear() datetime.daysinyear() date 1
dayofyear() datetime.dayofyear() date 1
firstdayofmonth() datetime.firstdayofmonth() date 1
weekofyear() datetime.weekofyear() date 1
hour() datetime.hour() date 1
minute() datetime.minute() date 1
second() datetime.second() date 1
millisecond() datetime.millisecond() date 1
nanosecond() datetime.nanosecond() date 1
offset() datetime.offset() date 1
gettimezone() datetime.timezone() date 1
getnumericdate() datetime.getnumericdate() date 1
gettime() datetime.gettime() date 1
len() struct.len() value 1
structcount() struct.count() value 1
arraylen() array.len() value 1
stringlen() string.len() value 1
xmlchildpos() xml.childpos() elem 1
xmlformat() string.xmlformat() string 1
xmlgetnodetype() xml.getnodetype() XMLNode 1
xmlsearch() xml.search() XMLNode 1
xmltransform() xml.transform() XML 1
5 Likes

Oh, yes please! Burn all those top-level BIFs for the core language!

2 Likes

Isn’t Hash always for a string? Not sure why it’s a special case.

Are you planning to leave BIFs such as pi() as just pi() - maybe a Math.PI is better?

this also removes the inconsistent placeholder orders of legacy BIF also. #yay

1 Like

Not exactly. Here’s the current annotations on our hash() BIF

@BoxBIF
@BoxBIF( alias = "Hash40" )
@BoxMember( type = BoxLangType.STRING, name = "hash" )
@BoxMember( type = BoxLangType.STRUCT, name = "hash" )
@BoxMember( type = BoxLangType.ARRAY, name = "hash" )
@BoxMember( type = BoxLangType.DATETIME, name = "hash" )
public class Hash extends BIF {

Meaning we support the headless names hash() and hash40() (for Lucee compat) and the member method .hash() on strings, structs, arrays, and dates.

Internally, the hash() function checks first if the input is a string, getting the byes via the supplied encoding. Otherwise, it checks if the input implements Serializable, getting the serialized bytes. And then finally, if the checks above don’t match, we call .toString() on the object and hash the bytes of that string.

So, at the end of the day, the hash BIF is set up to basically accept anything as input, so in order to go strictly to member methods, we’d need to allow it to be chained on every single object. Now, Boxlang’s method interceptor actually allows that pretty easily, but I don’t like that idea. I think that if a function is generic utility that can operate on any input, it should be a top level utility and not a type-based member method.

pi() does not operate on a number. It simply returns a constant value, so it’s not a member method by definition. So, for now, pi() stays as it is. Do I like the idea of something like Math.pi()? Yes, but that’s a conversation that brings a lot of questions about what BIFs it would involve and how we’d implement it, so I’d rather kick that can down the road for now.

I did ask people’s opinion on BIFs that operate on a number such as cos(). These CAN totally be member methods (and fit the definition). The question becomes an organizational one. Should every method that can be a member method be a member method simply for the sake of member methods? Like I said in the post, Java for example draws a line and puts stuff like cos() into a helper class instead of making it an endemic part of the number class itself. That doesn’t mean we have to follow Java necessarily, but I did want to have the conversation.

The main trade off I see is

  • The CF language for better or worse makes pretty much everything just “always available” without ever needing to import anything.
  • So if we carve off things into, say, a Math class, then now you need to import boxlang.core.Math or whatever any time you want to use it (more work/code trade-off for better organization)
  • ORRRR. we auto-import certain packages of classes which now has the side-effect of basically “reserving” certain variable names (like Math) in the same way BIFs reserve function names like cos() ) With CF’s runtime symbol resolution and pretty loose naming rules, we’re not used to having to avoid any particular words in our variable names.

Hrm. I think it’s less a case of “CAN totally be…”, and more a “if one squints, one can shoehorn it in there”.

These are trigonometric functions, so there’s a bit of a hint in the name there. I think they’re much more natural as static methods of a lib (class I guess).

One is not gonna want to go myAngle.sin(). No-one does that. It’s weird (to stick with the US elections zeitgeist). One is gonna want to go sin(myAngle). And if everything is to be a method, then it’s a static method on the class, not a behaviour of a number object. So: Math.sin(myAngle)


Oddly though, you’ve sold me on the hash(Serializable) then making sense as myStruct.hash(), myArray.hash() etc. Cos it’s not really a “you can call it on anything, just like how you can with a CFML function” (as per our perennial conversations on this topic), it’s more “you can call it on a Serializable” (where structs, arrays, etc implement Serializable).

At the risk (indeed, the liklihood!!) of being obtuse - could you expand on why? Not saying I disagree, just not understanding the benefit of being restrictive for its own sake; thus not allowing you to do something - albeit contrived - like:

x = myObj.getArray().hash()
and instead needing to do this:
x = hash( myObj.getArray() )

I guess we overlapped!

We’re used to Math.sin(x) because Java has numerics that are primitives (not objects) and they coerce in both directions (auto-boxing / auto-unboxing).

If all your numerics are objects, then member functions make sense and x.sin() is “normal”.

That said, I expected Ruby to have gone down that path… but it went down the Math.sin(x) path instead.

Java’s path is that Number has very few methods Number (Java SE 21 & JDK 21) (oracle.com) and even the more specific boxed types like Double only have instance methods that relate specifically to “boxed double” considerations or are static methods for manipulating the unboxed double (but, again, are specifically for double operations rather than generalized numerics).

Given that BoxLang very explicitly follows the Java path of having java.lang.Integer and java.lang.Double behind its numeric types, I think there’s a very strong argument for following Java’s model for Math (and perhaps adding the convenience of auto-importing it).

Note: the language purist in me would prefer everything to be a member function where it is possible and to have as few globals as possible – and as few helper-class-with-static-methods as possible too – but given that Ruby plays in that space and doesn’t go that path for Math persuades me somewhat that “numbers” are a reasonable (common) exception to that ideal…

Ah yeah, good point in the OOP sense of things. I still can’t see x.sin() as “normal” though (cf sin(x)), but quite poss due to it being unfamiliar as much as anything else.

Are there precedents in other languages for x.sin()? (and, yes, I know I could google this. Being lazy sorry).

Given that BoxLang very explicitly follows the Java path […] I think there’s a very strong argument for following Java’s model for Math

Also a good point.

That’s not the case at all. Like I showed above, BL already has hash() member methods on strings, structs, arrays, and dates (even though I think it’s questionable if producing a hash of itself is really something that falls in the domain of an array or a struct) and we don’t plan to remove that. You would still be able to call myArray.hash() as well as hash( myArray ).

The hash() BIFs behavior of basically accepting any input is what makes it feel like it deserves to still be a global function, but I never intended to remove the member methods from the other types.

Ah, my apologies - I suppose I misread. When you said “it should be a top level utility and not a type-based member method” I took that for removing that capability, rather than allowing for both.

1 Like
BIF Name Member Name Argument Name Argument Position
isempty() array.isempty() value 1
structisempty() struct.isempty() value 1
arrayisempty() string.arrayisempty() value 1

There might be typos in the “Member Name” for isempty() and arrayisempty()

1 Like

This is still on my plate, just back-burnered. I wanted to mention a wrinkle I had thought of in regards to struct BIFs. CFCs throw a bit of a wrench when it comes to getting rid of those. Any CFC instance can be treated as a struct (passed into any struct BIF) as in

q = new query()
structKeyExists( q, 'execute' )  // true

However, CFC instances do not have any inbuilt member methods, presumably to avoid conflict/confusion with actual methods on the class. This means the working CF code sample above cannot be rewritten as

q = new query()
q.keyExists( 'execute' )

and I don’t think it’s wise to “add” a struct member methods to CFCs. That means one of two things would need to happen.

  • We leave a headless Struct BIFs in BoxLang just for the use case of passing a CFC instance into them
  • We move all struct BIFs to some struct util class like so:
import ortus.boxlang.StructUtil;

q = new query()
// delegate to a util for struct treatment of a CFC instance
StructUtil.keyExists( q, 'execute' )

Or maybe it would be a class util. Not sure-- I haven’t thought that last one all the way through yet. Input on this wrinkle welcome.