Partition once versus filter twice for bulk email validation
The decision The partition function separates the array into the elements that satisfy the condition from those that do not docs . The filter function iterates over an array and applies an expression that returns matching values docs . Both scripts compute the task to split payload.records into accepted ids and rejected records with a reason, plus a retry count, by an email match. First approach The partition once approach uses the import line import * from dw::core::Arrays . %dw 2.0 import * from dw::core::Arrays output application/json var split = payload.records partition (r) -> (r.email default "") matches /.+@.+\..+/ --- { accepted: split.success map (r) -> r.id, rejected: split.failure map (r) -> { id: r.id, reason: "missing or invalid email" }, retryCount: sizeOf(split.failure) } Second approach The filter twice approach requires no import. %dw 2.0 output application/json var valid = payload.records filter ((r) -> (r.email default "") matches /.+@.+\..+/) var invalid = payload.records filter ((r) -> not ((r.email default "") matches /.+@.+\..+/)) --- { accepted: valid map (r) -> r.id, rejected: invalid map (r) -> { id: r.id, reason: "missing or invalid email" }, retryCount: sizeOf(invalid) } Same input, same output { "records" : [ { "id" : "ORD-1001" , "email" : "ana@example.com" }, { "id" : "ORD-1002" , "email" : "bad-address" }, { "id" : "ORD-1003" , "email" : "raj@example.org" }, { "id" : "ORD-1004" }, { "id" : "ORD-1005" , "email" : "mei@example.net" } ] } Both scripts print the same output for this input. { "accepted" : [ "ORD-1001" , "ORD-1003" , "ORD-1005" ], "rejected" : [ { "id" : "ORD-1002" , "reason" : "missing or invalid email" }, { "id" : "ORD-1004" , "reason" : "missing or invalid email" } ], "retryCount" : 2 } Measured Input Records Script Runs Min ms Median ms Max ms Source small 5 partition once 10 77 84 94 verified in sandbox large 50000 partition once 10 378 383 388 verified in sandbox small 5 filter twice 10 52 55.5 62 verified in sandbox