Unless I am defining things awkwardly, both patterns that I describe talk about service design.
I have been thinking about the question whether the coupling is tighter. I think it is, but not by much. The main difference in the direction of the relation.
I don’t have many examples, outside of what we are doing, of pattern #2, other than AWS lambda and similar services.
Thanks, it does have some of those characteristics. However in our case (a licensed solution / SaaS service):
Would you call AWS Lambda a closed platform? (Maybe I would, I’m just wondering if you would define that as closed)
edit: and the comparison with the iPhone / app store does is interesting. Not that what we do is comparable, but you are right that it does share some of the characteristics of what I’m trying to describe.
Ok I thought you were hosting it as well and there weren’t APIs but rather code and file folder patterns they have to follow.
I’m probably being unclear.
there weren’t APIs but rather code and file folder patterns they have to follow
There are APIs in our own design. Our middleware calls them all the time. They are also exposed when our users request it, but usually that’s because they want to do custom scheduling.
For a normal user they are hidden, and it’s about code and file folder patterns they have to follow
I dunno, still sounds frameworky to me (Wikipedia suggests the template method pattern as a name, btw):
In Django, you build a bunch of objects that implement the Django framework, then the Django code calls those objects to run a web server. Your users implement your interfaces and your code calls them to train models.
Clean Architecture has a chapter on frameworks that might be interesting reading, although it’s from the client point of view (and pretty wary of them).
Yeah I guess it’s close enough you could call it framework vs. library. A library supplies a bunch of methods that the client application can call and use how it wants. A framework is shared code that runs that client application in a very particular way.
In general a library is preferable if your application devs are competent and independent. But if you need 100% reliable behavior and maybe aren’t sure the application devs know what they’re doing - a framework might make more sense. (I’ve written both)
I’ve done stuff in React, which is supposedly a library, and Flask, which is supposedly a framework, and I’d be moderately hard pressed to explain why they are supposedly different.
I think as a profession we worry too much about what to call things and then go and come up with names like Splunk.
Yeah those things are so multi-headed it’s hard to tell.
I guess the react guys would say it’s still up to you to build the web site and just incorporate react features - but it’s a pretty fine distinction. Create-react-app would maybe be considered the framework that runs the react library - if you choose to use it.
I think one common distinction in practice is that a framework has mandatory updates which somehow need to be pushed out to all the clients to keep everyone in sync. IE - a framework supplier can dictate terms that they never have to support two major versions of the framework at once. (Which we did because our framework consumers were internal.)
Whereas a library is up to the client when they want to pull the update. Multiple major versions of the library can and should be supported at the same time. A library can be deprecated, but good practice is that it will still work for a while before being fully shut down.
But a framework could support multiple versions too I guess.
We had a big debate over library vs. framework when I built our node orchestration layer back in 2013-ish. Lib is preferable. But knowing the level of comfort our devs (half of which are offshore) had with node, I went the framework route. We knew we’d need to constantly implement cross-cutting concerns at multiple places in the request-response middleware chain (like say a new kind of auth for a certain call, or reporting). I did not want a whole bunch of special snowflakes implementing a lib in 20 different ways.
This is it btw: GitHub - jackspaniel/yukon: Self-discovering datasource-agnostic web components Apparently it still runs the middle layer (between the angular front end and scala/play API) of the directv entertainment site. https://www.directv.com/guide?lpos=Header:1
As a framework it says “Create components that look exactly like this and the framework will find them and implement them as an endpoint to the front end, with a defined back-end API call(s) in parallel (our API was set up that way - I made another version of the framework that can handle a mix of parallel and sequential back-end calls). You can implement any business logic you need before or after the call if you put it in methods with these names.”
As a lib it would be something like “Here’s some convenience methods to call our exact APIs in parallel. Other than that implement the request end points and the system-level, and individual request-response business logic however you want.”
I think in general a framework when your clients are 3rd parties and not internal sounds like it would be very hard to do.
Thanks all, this really helps.
Library vs framework does come really close. I’m going to read up on it.
Yeah I guess it’s close enough you could call it framework vs. library. A library supplies a bunch of methods that the client application can call and use how it wants. A framework is shared code that runs that client application in a very particular way.
In general a library is preferable if your application devs are competent and independent. But if you need 100% reliable behavior and maybe aren’t sure the application devs know what they’re doing - a framework might make more sense. (I’ve written both)
This is a very good summary of why we choose this route. Many data science teams, while they are filled with pretty damn smart people, shouldn’t need to know, and often don’t know how to produce something close to production-ready code when it comes to containerization, deployment, infrastructure & resource management, version control, etc. And many organizations, especially strictly regulated businesses, need 100% predicable behavior from their ML pipelines.
haha, for sure.
It does help to have a shared vocabulary when talking about these things, though.
For data science pipelines, a framework is appropriate.
Another keyword is event driven. The framework calls me.
Hoping some of the Excel gurus made the jump from 22.
I have a series of charts with slicers, to allow users to drill down in a dataset without having to display dozens of charts. Having the slicers forces us to store the data in an Excel table structure.
The problem is that whenever ANYTHING changes in the table — data, row headings, adding rows, whatever — ALL of the charts revert back to their original formatting, and we have to change them all one at a time. This is unsustainable as we will be updating the data tables on a monthly basis.
Anyone have any experience with this? Any simple fixes?
Got a problem here - I have a list of objects, doesn’t matter what they are. Some of them satisfy a condition and some don’t. What I want is a list of lists, containing all the contiguous sections of objects which satisfy the condition. In other words it’s a gaps and islands problem.
To maybe make this clearer, say I have this list of objects, with Y meaning it satisfies this condition and N meaning it doesn’t:
NNNYYNNNYNYNYYYYYYYYYYYYYYNNNNNNNNNYN
What I would want as output is this:
{ YY, Y, Y, YYYYYYYYYYYYYY, Y }
Of course I can do this with gross code festooned with loops and temp variables, but I’m hoping there’s a clean functional way to do it. If anyone can describe a solution conceptually I should be able to translate it into Java Streams.
Loop over the data structure with a temp variable list.
For every Y enter into the list
For every N if list non empty add it to set of lists and set list to empty
This is O(n) so you should be good to go
Yeah pretty much. And yeah it’s not a string, but I do want something along the lines of split.
Yeah I was hoping to have to avoid this sort of messiness, thought there might be a one line solution, but the solutions I’ve found so far haven’t been any more readable than this.
My boss just told our PM team that devs are too busy to do sprints or provide status reports and that PMs should just read the jira tickets to see where the very high-level tasks are at. Lol.
Our PM team and dev team have two different bosses who hate each other. It’s such a dumb annoying set up.
Yeah I don’t think you’re going to do better than that. It’s optimal in time complexity and it’s not common enough to have a one liner type approach. Best thing to do is make it a function that takes in an arbitrary list of objects and a mapping function that says Y/N, so it’s not plaguing up the rest of your script.
const list = [1,4,5,6,6,7,2,4,6,7,7,2,3,1,1]
const output = list.reduce((acc, obj) => {
const subList = acc[acc.length-1]
if (obj % 2 === 0) subList.push(obj)
else if (subList.length > 0) acc.push([])
return acc
}, [[]])
console.log(output); // [[4], [6, 6], [2, 4, 6], [2], []]
This probably isn’t anything like what you want - but just for fun I did it with reduce in JS. I used even/odd to simulate the condition.
Only problem is it can leave one empty array at the end sometimes if the last item(s) on the list is odd. Easy to strip off at the end I guess.