|
I do similar but 3x weekly, I run robocopy with /MAXAGE:8 to a directory like /evenmonths/2016/Dec/Week51, on a Samba server. Monthly gets the last 2 years and yearly gets it all. Small data share, only about 100GB so it is all manageable and can keep for the whole year and then shelf it. Weekly backups are about 200 meg. Monthly stuff is 4 servers (VM) via Veeam along everything needed to recreate all servers, then taken off site. Veeam will also restore individual files/folders. I run the free version with a script and back up (6) VM's twice a month. Like you, all are also copied to external drives. I use RDX that are ejected by the VBS script after the copy over.
We issue new tin foil hats monthly.
|
|
|
|
|
If I understood your post correctly you don't have a backup issue, it's an application issue. If your application writes crap, you'll backup, ahem, no offense, crap:
the file is physically "ok", in the sense that it can be read and copied without any sort of file system error--it's the software that last wrote to it that managed to write bad data somehow and made it unusable--the content can no longer be parsed, and in fact the size went from a few hundred MBs down to a few dozen KBs. Clearly, something's gone horribly, horribly wrong and this file is simply no longer usable. There's nothing in it to recover--no need to even try.
If I'm correct, I have some ideas, comment back.
cg
Charlie Gilley
<italic>Stuck in a dysfunctional matrix from which I must escape...
"Where liberty dwells, there is my country." B. Franklin, 1783
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
|
|
|
|
|
That's correct. Sure, the application isn't blameless in this case, but still, it proved that my backup strategy is somewhat lackluster.
modified 3-Dec-16 22:08pm.
|
|
|
|
|
Well, I guess my point is that backups backup, they have no idea what they are backing up. Many of the applications I've developed over the years require writing files and moving them around to other machines. File movement can be done over ftp or via USB (embedded environment). The surest way to make your life miserable is to trust the file(s).
It's just amazing how frequently these files are corrupted due to an ftp burp, variations in USB drives, evil users, etc. As a developer, you just don't think the corruption will happen - we tend to know not to remove the USB early, don't edit the files, ....
So, all of our files contain a CRC value. Files copied to USB are then read back and validated. It might be you need to do something like this.
Funny story - I once opened up a support ticket asking "How do I know my backups are good?" This was interpreted by IT as "Why are my backups not good?" and it sent them into a tizzy. Too much caffeine that day I guess.
Charlie Gilley
<italic>Stuck in a dysfunctional matrix from which I must escape...
"Where liberty dwells, there is my country." B. Franklin, 1783
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
|
|
|
|
|
charlieg wrote: So, all of our files contain a CRC value. Files copied to USB are then read back and validated. It might be you need to do something like this.
Not sure if this is what you're getting at, but I never claimed I wrote the app that corrupted the data.
And frankly I know exactly how it happened, and it was a probably my own fault (although the app could have been doing more sanity checks.
The data file was sitting on the NAS, which is a Windows 7 machine handling little more than file shares. The app that was using the file is running on another machine on the network. I had to reboot the NAS machine after installing updates, and I allowed it to, forgetting that this one file happened to be opened on another box. I remember intentionally avoiding saving while the machine was rebooting, but after it was back up and confirmed with Explorer I could again access shares, I hit Save as I had changes that needed to be saved. While the application didn't complain - and I guess I was shortsighted enough to think everything was ok - that was the last time I've seen any of the data in that file. At the very least I should have saved to another file, and immediately verify its integrity by closing/reloading it. I guess despite what other say about me, I'm still too much of an optimist...
|
|
|
|
|
 "too much of an optimist"
I resemble that remark. I understand what happened now. Falls under, "I should have been more careful."
Charlie Gilley
<italic>Stuck in a dysfunctional matrix from which I must escape...
"Where liberty dwells, there is my country." B. Franklin, 1783
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
|
|
|
|
|
Am I a good developer ? This question bugs me at times. On one hand I am always able to provide solution to business problems using what I know or by learning something new to resolve problem. On the other hand I don’t know all the latest and greatest frameworks and tools out there.
I have been doing development for more than 7 years now. In those 7 years I have worked on so many different tools, technologies and libraries to solve business problems. I have .NET as my main technology stack but I did development on Java, SQL, Integration tools, Salesforce, PLC and few other technologies. One thing is consistent in all those projects is business generally don’t care about underlying technology stack as long as it serves their requirement. As a developer you make conscious decision on what to offer to your client as a solution. You cannot ask a small shop to go on and invest arm and a leg on some product just because you are comfortable working with it and on opposite side there are big organizations which can afford to buy or build products using latest and greatest technologies. As a developer you understand that latest is not always the best. Why would you implement a solution using some new JavaScript framework or any other latest fad that takes more time and effort and eventually becomes a maintenance nightmare? Just because Google and Facebook uses Python I must use it ? As a developer you are not only required to know programming languages but also different server platforms, deployment tools, source control tools, CI / CD platforms, testing frameworks etc. A developer’s job is a demanding job.
In today’s market there is ever growing expectation from developer to be proficient in whatever technology is hot at that time and there are quite a few at any given point in time. Not only you need to know that technology inside out you are expected to know any other tools, libraries and frameworks build around it. It makes you question are you really a good developer even if you don’t know latest technologies or tools ? You don’t know those technologies because up until now you did not have a need to use it in whatever application you are building. Just because AngulrJS sounds cool and everyone is using it I must use it ? One thing I have felt is if you don’t use today’s latest fad you are perceived to be old school and at times will not get a chance to even be considered for a new role.
Do you ever wonder are you a good developer or not ?
Zen and the art of software maintenance : rm -rf *
Maths is like love : a simple idea but it can get complicated.
|
|
|
|
|
I think I'm a good developer, but I know I have lots more to learn, both in breadth and depth of knowledge. Thankfully I work with a great bunch of people who are much smarter than I am and don't have egos. I like that my work is reviewed by people brighter than myself, because that enables me to grow as a developer.
Because my interests and the nature of my job center around Microsoft technologies (specifically .NET server and desktop apps) and Android, I focus on these technologies. I don't have a desire to learn web front-end technologies like JavaScript (and its large associated family of platforms), even though they're very popular and in high demand.
I'm grateful that I love my job and career and wouldn't change it for anything.
/ravi
|
|
|
|
|
virang_21 wrote: Am I a good developer ?
Hi, Virang,
In the rare moments I find myself thinking about "how good a ... 'whatever' ... I am," I try to (gently) acknowledge that, and re-direct my attention to what I am doing/learning in this moment.
It is more important to me to have a sense that I am on the right path to being a better 'whatever.'
In terms of what skills, and accomplishments, will get you hired in today's marketplace, I tend to believe that "narrow-deep and exuberant willingness to learn" is, inherently, stronger than "wide-shallow." Your mileage may vary.
Perhaps it's too easy for me to say these words, because I am (relatively) older, don't need to work, and I don't have that many programming-cookies in my identity-jar at this point in my life ?
best wishes, Bill
«There is a spectrum, from "clearly desirable behaviour," to "possibly dodgy behavior that still makes some sense," to "clearly undesirable behavior." We try to make the latter into warnings or, better, errors. But stuff that is in the middle category you don’t want to restrict unless there is a clear way to work around it.» Eric Lippert, May 14, 2008
|
|
|
|
|
In my opinion, the moment you stop asking yourself this very question is the moment your question is answered. Development is a near infinitely complex, ever-evolving career. The moment you believe you know it all and stop learning is the moment you fall behind and become obsolete.
|
|
|
|
|
virang_21 wrote: Do you ever wonder are you a good developer or not ?
Good? Never! How about lawful evil?
Seriously, concentrate on things that don't ever come out of fashion, like architecture or algorithms and pick up from a project what you can. After all, nobody needs to be able to do every job. One at a time that fits is enough.
The language is JavaScript. that of Mordor, which I will not utter here
This is Javascript. If you put big wheels and a racing stripe on a golf cart, it's still a f***ing golf cart.
"I don't know, extraterrestrial?"
"You mean like from space?"
"No, from Canada."
If software development were a circus, we would all be the clowns.
|
|
|
|
|
CDP1802 wrote: Good? Never! How about lawful evil?
He's a programmer, not a manager 
|
|
|
|
|
Sander Rossel wrote: He's a programmer, not a manager
Managers aren't "lawful evil" - they tend to "Awful Chaotic", or "Chaotic Evil" at their worst. 
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
|
|
|
|
|
A manager must be chaotic evil.
The language is JavaScript. that of Mordor, which I will not utter here
This is Javascript. If you put big wheels and a racing stripe on a golf cart, it's still a f***ing golf cart.
"I don't know, extraterrestrial?"
"You mean like from space?"
"No, from Canada."
If software development were a circus, we would all be the clowns.
|
|
|
|
|
Most manager are chaotic neutral. Any inherently good or evil actions are simply a lack of self control and occur by accident.
|
|
|
|
|
I learned very early in my career that I was forever meeting and working with people who were far better developers than me. Which was a good thing because it gave me levels of ability to strive towards. But I never considered myself a good developer, just that occasionally I managed to produce some good code. Unfortunately I continued to make many mistakes.
|
|
|
|
|
If you were a mechanic, would you question your ability because you aren't skilled in all areas: general repair, electronics, small engines, diesel, foreign imports, jets, planes?
Probably not.. you'd have your areas that you were capable in and work from there; you'd probably work on expanding your skill set, but I don't think you'd expect you MUST be able to be an expert in all areas.
So... why do we, as developers, think that we need to keep current with an ever changing playing field?
My career started with Fortran, VAX/VMS and FMS Forms; it grew to include OSISoft PI systems and then C and Ingres. Next up was Windows development, SQL Server and Oracle.
Now.. I work primarily with OSISoft based technologies - I touch Windows development and SQL Server, but only in that they are required for the core technologies I use.
|
|
|
|
|
virang_21 wrote: In today’s market there is ever growing expectation from developer to be proficient in whatever technology is hot at that time and there are quite a few at any given point in time. Not only you need to know that technology inside out you are expected to know any other tools, libraries and frameworks build around it. It makes you question are you really a good developer even if you don’t know latest technologies or tools ?
This is mixing apples and oranges.
For the TLDR folks:
Good developer: overall skills and technical knowledge in specific areas capable of learning new things.
Good fit: a person with the specific skills the business needs now.
Those are two different things.
Longer read:
First off, I'm a good developer because I have a lot of (what I think is) very good overall knowledge, practices, work ethic and discipline, including skills such as communication, documentation, thinking outside of the box, and so forth. I'm also a good developer because I can learn new things -- business processes, technologies, etc. I'm also a good developer because when I start on a project, I ask a lot of questions, I assume I know nothing, I ask why decisions were made that are contrary to how I would have done things, so I can learn what the trade-offs were, and I constantly look at what I'm doing and work on self improvement.
On the other hand, if a client / employer needs someone with a specific expertise to immediately jump in and work on something in which I don't have those specific technical skills, then I am not a good fit for their requirements. That does not mean I'm not a good developer. It only means that for their business requirements, they can't afford the (sometimes considerable) tooling up / training that I would need.
Obviously, there is the third camp, let's call those people junior developers, that are neither good developers (and hopefully will be one day with proper mentorship) nor have the technical skills. Those are the people that companies are the most reluctant to hire nowadays. The idea of grooming someone just out of school, building a long term relationship with them, having incentives for their staying with the company, etc., those days are for the most part long gone, partly because of market forces and partly because companies more and more treat people like replaceable cogs in the wheel, when actually they are not, especially once they've learned a lot of the intangible knowledge of the specific business domain. I'm not saying anything original here, this issue has been written about for years.
[edit]
The other thing about modern development, and something that seems to be not fully recognized, or worse, ignored, is that the days of one man development are pretty much over. It really does require a team nowadays - front end, back end, middleware, source control, documentation, testing, QA, deployment -- and in that world, while it's still important to know your particular area of expertise well, communication, organization, and "higher level" skills are becoming (IMHO) even more important, and also what I find generally lacking are the people that can glue it all together well. You might call these people "architects", but they are certainly not the typical product/project manager.
[/edit]
Marc
modified 3-Dec-16 11:04am.
|
|
|
|
|
Eloquent, as expected, Marc: I like the focus on "fit;" I wonder if that might be extended to thinking of hiring (at its most "rational") as being a cost/benefit analysis which weighs immediate fitness, potential fitness, and long-term adaptability in an industry where hardware, and software tools, will change radically every few years.
I spoke today with a former colleague (in the U.S.) from my years at Adobe; he's 59 years old, has a stunning resume: he says he's unemployable right now in Silicon Valley, and he believes that his age is a factor.
Did you ever read Coupland's book, "Microserfs" ?
thanks, Bill
«There is a spectrum, from "clearly desirable behaviour," to "possibly dodgy behavior that still makes some sense," to "clearly undesirable behavior." We try to make the latter into warnings or, better, errors. But stuff that is in the middle category you don’t want to restrict unless there is a clear way to work around it.» Eric Lippert, May 14, 2008
|
|
|
|
|
BillWoodruff wrote: Eloquent, as expected
High praise, coming from the man that defines eloquence!
BillWoodruff wrote: he's 59 years old, has a stunning resume: he says he's unemployable right now in Silicon Valley, and he believes that his age is a factor.
I wonder about that, being 54. I wonder about the CTO of my former employer saying "I'm going to hire a bunch of young junior developers." It's rather shocking that a person in SV with tons of experience can't find a job!
BillWoodruff wrote: Did you ever read Coupland's book, "Microserfs" ?
No, but it looks quite interesting. I'll put it on the reading list.
Marc
|
|
|
|
|
I agree, Marc's post hits a number of critical points.
BillWoodruff wrote: I spoke today with a former colleague (in the U.S.) from my years at Adobe; he's 59 years old, has a stunning resume: he says he's unemployable right now in Silicon Valley, and he believes that his age is a factor.
This raises a key point, that there are two sides to being a developer: For the employer/customer and For yourself.
These are quite different. Being a good developer for the employer/customer means being the person who (typically) works with a team to solve business problems, using all the skills and abilities that have been previously noted.
How do you be a good developer for yourself?
By staying marketable.
We have to remain in "education mode" for our entire careers. Technologies come and go, so we have to learn new things to remain employable. I've seen too many people ride a technology into its oblivion, then wonder how they are going to find another job since they don't have other skill sets.
Technology is like the Lernaean Hydra, chop one head, at least 2 more grow from the stump. It's not possible to learn it all, so we have to make informed decisions on what to pursue. That gets easier as we gain more experience, we learn how to better choose which technologies to pursue. [Corresponding with others online certainly helps in gaining perspective.]
Couple this with improving the non-technical skills like problem solving, business astuteness, listening to customers, etc. This keeps us marketable.
Is this really different from being a good developer for the employer/customer?
Yes -- the difference is a matter of focus -- one side is for others while the other is for ourselves. They converge in the same place (being a good developer), but without devoting time to our own needs (marketability), we may end up without a marketable skill set.
Regarding Bill's friend who can't find a job in Silicon Valley -- ageism is a separate issue. There are places that are happy to hire the experienced people. Maybe not the "hot" employers ... but I have had several jobs in recent years where no one under 50. The client wanted experienced people who did things right on the first try. Unfortunately, Bill's friend might be forced to pull up roots and move to find such a place. After years of living in a place, that can be rough. 
|
|
|
|
|
virang_21 wrote: In today’s market there is ever growing expectation from developer to be proficient in whatever technology is hot at that time and there are quite a few at any given point in time. Not only you need to know that technology inside out you are expected to know any other tools, libraries and frameworks build around it.
I'd rather not work for someone who has those kinds of unrealistic expectations. I'd rather work for someone who values one's ability to learn and adapt to any new technology being thrown at them.
That is, by far, the more valuable skill.
|
|
|
|
|
virang_21 wrote: In today’s market there is ever growing expectation from developer to be proficient in whatever technology is hot at that time and there are quite a few at any given point in time. Not only you need to know that technology inside out you are expected to know any other tools, libraries and frameworks build around it. It makes you question are you really a good developer even if you don’t know latest technologies or tools ? You don’t know those technologies because up until now you did not have a need to use it in whatever application you are building. Just because AngulrJS sounds cool and everyone is using it I must use it ? One thing I have felt is if you don’t use today’s latest fad you are perceived to be old school and at times will not get a chance to even be considered for a new role.
The problem, I think, is that people tend to equate libraries and frameworks with technology. They're not. They're approaches to problem solving at best, and like all approaches they involve trade-offs.
The bigger question is: can you write good, stable, and functional code without the new toys, understand what frameworks are doing, and play nice with them by doing a careful code-walk (assuming they're not a steaming pile of garbage, which many are).
So there's an easy answer: the only objective measure of if you're a good software developer is if the software that you develop is good. That's the only thing that matters; libraries and frameworks are just tools to achieve that end, and that's all that they are.
"There are three kinds of lies: lies, damned lies and statistics."
- Benjamin Disraeli
|
|
|
|
|
I am paid well, so I am a good developer.
|
|
|
|
|
I'm paid adequately, so I'm a 'good enough' developer.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
