|
|
Hi,
I'm working on a new design for our old (25+) application. In this design should be more than one SQL servers (synchronized via replication). Each SQL server wrapped inside a DAL layer, and those DAL's are grouped using load balancing.
I looking to add cache to this design, and at first I thought that the best place to do so is at the individual DAL, however in this case I have to design a synchronization method between separated DALs.
To solve this synchronization problem I thought about a cache service to serve all the DALs (and maybe other parts of the design).
My question is, according to your knowledge and experience, will it be still effective to use a remote service to cache, or better to design cache synchronization that cross DALs?
Regards,
Peter
|
|
|
|
|
That question is hard to understand.
Persumably you have 25 or more applications. And each of those use 1 or more databases.
After that your explanation loses me.
Do you intend, at some future time to consolidate databases? And that is a hard plan driven by business reasons? Because if not then they is absolutely no reason why any of these should be combined into a single cache.
Is there a distributed transaction model in play? If not then there is no need for "synchronization". And if there is then you should look into an actual distributed transaction model.
And why do you think you need to cache everything? Or even anything for that matter?
And I have no idea what you think "load balancing" means in this context. That term means a way of balancing requests across different servers. A DAL (Data Access Layer) exists within the application and unless all of the applications run on one machine load balancing across multiple applications would be difficult.
|
|
|
|
|
First of all thank you for your time...
and now some explanations to make it clear...
There is only one app - a web one. The application is distributed so the DAL is only a part of it (there are many layers between the UI that sits in the web server and the DAL which is combined, if not necessary physically, with the actual DB). The DAL is designed in a stateless way so we can lunch infinite number of DALs. In case of multiple DALs they grouped with load balancing. And when I say load balancing I mean exactly what you mean...
I thought about cache to improve performance of the DAL, but after looking into it I saw that I must
or make some synchronization between local caches
or make some cache server.
My question was about the usefulness of such cache server...
Regards
Peter
|
|
|
|
|
Kornfeld Eliyahu Peter wrote: The DAL is designed in a stateless way so we can lunch infinite number of DALs
Then you have a server, not a DAL.
Kornfeld Eliyahu Peter wrote: My question was about the usefulness of such cache server..
Depends on the data and the nature of the business.
If, for example, you have some small set of data that is used a lot and doesn't change often then caching is doable even with the complication of cross server syncing. As long as there is some allowed latency in the timeliness.
Conversely if you are loading billions of customer records by request from a user then there isn't much point because each request by itself likely has a very limited lifespan in the cache. And you would also need to implement sticky sessions for it to be useful.
|
|
|
|
|
We're developing a couple of web applications and want to allow users some advanced options if they've identified themselves.
The goal is that the user should only remember one username/password for all our applications (and services, we provide all kinds of newsletters and alerts as well).
SSO (Single-Sign On) was the first thing that came to mind, so my question is: what kind of recommendations can you give? I read a little about OpenId, but I know Google, Yahoo, windows live, ... also provides this.
Should we choose an existing service, and wich one is the best or should we write something for ourselves for our company only?
In the (near or further) future I would like to add the personnel as well through ldap or something.
This stuff is completely new to me so any advice, tutorials, recommendations would be helpful.
Thanks!
|
|
|
|
|
I have not a huge experience with SSO but just from how I'd feel as a user I'd say using an existing service as only possibility would be not a great idea.
If you us another service then you should add your own possibility too, such as CodeProject does offer a merged sign-on for codeproject.com and rootadmin.com - combined with the chance to use a Google account for signin in/up.
|
|
|
|
|
To the best of my knowledge you wouldn't be wrong to take a look at a form of Federated Identity Management using a Token Service. OpenId, SAML, WIF and OAuth are all token-based and will take you down the road of claims-based authentication and authorization.
http://en.wikipedia.org/wiki/Claims-based_identity[^]
I would have used something like STS as a starting point for a token service, but our management in their infinite wisdom want us to roll our own token service. This despite the fact that our token service will not be interoperable with anything else as it doesn't support any common standards beyond putting a token on the same HTTP header as other token services do. Oh, and there's no integrity check for our claims and everything is passed as clear text. Well, not as clear text actually, we're base64 encoding the token so it would only take a determined person a couple of extra seconds to walk right on in. Then there's the issue of token size, which is limited, so we'll roll our own zip function to cope with that, even though a decent token service will already do this for you, along with everything else we've implemented for no good reason.
But whatever, rant over. Just don't try and reinvent the wheel like our place does. Token authentication is not a walk in the park by any stretch of the imagination so anything you can use off the shelf will save you a ton or arseache.
STS: http://startersts.codeplex.com/[^]
|
|
|
|
|
|
Hello,
When I ask questions about architecture I often hear back several arguments to explain this or that choice.
arguments are
* For maintenance
* For performance
* for Security
* For data consistency
What, for you, are valid arguments to make a decision?
Personally I place "data consistency" in number one and it is a non negotiable priority but I think I'm alone to think like that because the current pattern, decisions, modern architectures emphasize maintenance, performance and security.
|
|
|
|
|
Hmmm. That's an interesting question. I would agree that consistency is an important consideration, but how do you define consistency? I ask this because you really need to put a qualification on consistency so that you have a measurable baseline. Unfortunately, this isn't a case of black and white - do you mean instantly consistent (which has a huge impact on your database architecture) or do you mean eventually consistent (whereby you could have an update made from one server and know that this would eventually be updated in all the other servers). There is no hard and fast answer to this question.
|
|
|
|
|
By data consistency I suggest using massively normalization, constraints and triggers in SQL database. This may be very hard to maintain and many architects doesn't like that. I also design my database without duplicate any information. For performance issue some architects already ask me to repeat a column to avoid a join.
|
|
|
|
|
That isn't data consistency. That's just fundamental design. Data consistency is a much broader area - it relates more to things such as transactional consistency or point-in-time consistency.
|
|
|
|
|
I agree with you data consistency is not only my example. From Wikipedia "Data consistency summarizes the validity, accuracy, usability and integrity of related data between applications and across an IT enterprise".
In case of a SQL database data consistency may be assured by transaction, constraints and triggers. In fact I don't see other way to assure data consistency in a SQL database. Now adding trigger, constraints and complex transaction impact performance, security and maintainability.
modified 13-Mar-13 6:22am.
|
|
|
|
|
The example you give here would normally come under transactional consistency - the ACID rules if you like. However, it doesn't take into account things like data replication strategies, load balancing and so on.
Take Code Project, for instance. I know that Chris and team have tuned the system as well as they can, but there are still delays with updates simply because of replication. You vote on a post and refresh the page - the vote doesn't appear to be there - but that's because you are now looking at a different server. All you can say is that the data will eventually be consistent.
|
|
|
|
|
I understand you example. It's more about deployment or infrastructure architecture. Data may be inconsistent in time only. But the 4 arguments I described are still valid for deployment or infrastructure decisions.
In fact by data consistency I'm talking about avoiding to insert wrong data in the database. I don't trust my developers  . I need to protect my data but think about performance and maintainability. My database has been designed with redundancy. I already noticed my developer forget to update the redundant data when they modify the first one. I would like to force them to do it right but I can't create what I want for maintainability reasons.
|
|
|
|
|
DranDane wrote: What, for you, are valid arguments to make a decision?
Since most of the time, maybe even all, the actual "arguments" are based on subjective opinions it doesn't matter much.
DranDane wrote: Personally I place "data consistency" in number one
I place the actual business needs as number one. Banks probably (hopefully) place security and consistency high. Where a mom and pop retail store probably places cost (which you didn't list) as the highest factor.
|
|
|
|
|
You right. It to the business to choose what is the most important priority for him
|
|
|
|
|
I've been looking at messaging systems like Apache's [Qpid] and [RabbitMQ], but I'm wondering what any of you folks are using for cross platform development messaging?
Do you build mobile code that calls existing/standard web services (SOAP/WCF) that can also be utilized by existing desktop applications, or is there another way?
What I'm trying to understand is the best way for a set of mobile applications and desktop applications to be able to communicate (in regards to shared services) to send/receive data across these disparate systems. We're typically building RESTful services, but allowing mobile to call existing services is our primary goal. Secondary is abstracting the messaging away from source specific knowledge, preferably with an established third party messaging engine.
Thanks.
|
|
|
|
|
Mobiles are typically client applications and client applications might send messages but they don't typically receive them (for good reason.) A client app would send a message by calling a server. Because of that it means that mobiles have nothing to do with it.
Once you eliminate the mobile apps exactly what platforms are you left with?
dexterama wrote: Secondary is abstracting the messaging away from source specific knowledge,
preferably with an established third party messaging engine.
Messages are data so language is almost never an issue in terms of the message. Typical problems that show up with language is when someone attempts to insert a language data structure into a message stream and doesn't understand that it is being serialized.
Other than that your requirement really says nothing. It doesn't even suggest that you actually need a messaging system.
So you might want to work on some actual business requirements first and then try to find a technology that will meet the needs of those requirements. Be sure to also collect information about volume and expected realistic growth rates for that volume.
|
|
|
|
|
Quote: Mobiles are typically client applications and client applications might send messages but they don't typically receive them (for good reason.) A client app would send a message by calling a server. Because of that it means that mobiles have nothing to do with it.
Messages = data, so yes, mobiles get message packets all the time. My mobile real estate app gets a packet of items it displays in a grid, as does Amazon, eBay, etc.
Think of how stupid the average person is, and realize half of them are stupider than that. - George Carlin
|
|
|
|
|
dexterama wrote: Messages = data, so yes, mobiles get message packets all the time.
Initiated by a call on the client; the server does not initiate the message. It just "serves" the messages that the "client" requests.
Bastard Programmer from Hell 
If you can't read my code, try converting it here[^]
|
|
|
|
|
dexterama wrote: Messages = data, so yes, mobiles get message packets all the time
A 'client-server' system in the modern world runs on TCP (over IP) at that means one side is ALWAYS the client and one side is the server. This is true regardless of how many API layers you stick on top of it.
In a message framework system, there are publishers and subscribers how that does not specify HOW the data moves. All it does is describe how the messages SEEM to move. Since it MUST use TCP that means that there will ALWAYS be a client and a server, still, in the message framework. And what that means is that typically a client app will act as a client (which shouldn't be surprising) and some other system will be the client.
However message frameworks can in fact be set up such that the client app is a server.
So with that background...it is seldom going to be a good idea for a mobile app to be a server. Nor for client apps of any sort to be a server.
Additionally just with any complex system it probably isn't a good idea to expose it directly to the internet. Which would be required if the mobile app was receiving pushes (which means it is acting as a server.)
Is that clearer now?
|
|
|
|
|
Hi
I have 2 application servers running my web application.
Application has a feature which upload a file. Now file upload is on server file system
There are 2 ways of doing this one is either have a file server where all files are uploaded so that whenever application is accessed all files are visible.
Other is to sync between 2 servers for files uploaded either through service or manually [clicking a button]
I like file server but then it will involve extra cost of setting up server and permissions on the folder is there any other way we can achieve this?
|
|
|
|
|
nitin_ion wrote: is there any other way we can achieve this?
You could set up a data partition on one of the both servers and set up a FTP service or whatever else you want there. In the end you'd have a file slave server which is uploading the data to the file master server. The file master server has access to the additional shared partition too and lays the files there down.
However - a dedicated file server would be the most secure way of doing this because one web server has access to the files even when the other web server is down, therefore I strongly recommend you to use a dedicated file server.
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
