Related
I'm thinking about the possibility of teaching the use of Software Transactional Memory through 1 or 2 guided laboratories for a university course. I only know about Haskell's STM, but the students of the course probably never heard a word about it.
I already found some lists of such libraries online or in other questions (e.g., http://en.wikipedia.org/wiki/Software_transactional_memory#C.2FC.2B.2B). I'm checking them out as you read this, but many of them do not seem to have a very nice documentation (most are research prototypes only vaguely described in papers, and I would rather teach about something more used and well documented).
Furthermore, many of the links provided by wikipedia are dangling.
To sum it up, are there STM implementations aimed to industrial projects (or at least non-toy ones, to ensure a certain level of quality) and well documented (to give some good pointers to the students)?
EDIT: I'm not the teacher of the course, I just help him with the laboratories. Of course the students will be taught basics of concurrency and distributed algorithms before. This was just an idea to propose something different towards the end of the course.
Production-quality STM-Libraries are not intended as a teaching tool, not even as "best practice". What is worth learning for any college/university-course is maybe 1% of the code; the remaining 99% is nitty-gritty platform-dependent intrinsic corner-cases. The 1% that is interesting is not highlighted in any way so you have no way of finding it.
What I recommend for a college/university-course (no matter if introductory or advanced) is to implement STM-buildingblocks yourself (and only for 1 platform).
Start by introducing the problems: concurrency, cache...
Then introduce the atomic helpers we have: cas/cmpxchg, fence.
Then build examples together with your students, first easy, then harder and more complex.
Start by introducing the problems: concurrency, cache...
Leading on from eznme, some good problems that I covered while at University for concurrency.
Dining philosophers problem
In computer science, the dining philosophers problem is an example problem often used in concurrent algorithm design to illustrate synchronization issues and techniques for resolving them.
(source: wikimedia.org)
Using the same implementation from here, by Je Magee and Je Kramer, and solving the problem using monitors.
Most shared memory applications are more efficient with Integers than Strings (due to AtomicInteger class for Java). So the best way to demonstrate shared memory in my opinion is to get the students to write an application that uses a threadpool to calculate prime numbers, or to calculate some integral.
Or a good example of threads and shared memory is the Producer-consumer problem.
The producer-consumer problem (also known as the bounded-buffer problem) is a classic example of a multi-process synchronization problem.
(source: csusb.edu)
Implementation found here, there is also an implementation from Massey from the professor in Software Eng Jenz Dietrich.
For distributed algorithms MapReduce and Hadoop are highly documented distributed data structures. And for Distributed Programming Libraries look into MPI (Message Passing Interface) and OpenMP (or Pragma for C++). There is also implementations of Dijkstra shortest path algorithm in parallel too.
There are three good ways to do STM today.
The first way is to use gcc and do TM in C or C++. As of gcc 4.7, transactional memory is supported via the -fgnu-tm flag. The gcc maintainers have done a lot of work, and as of the 4.9 (trunk) branch, you can even use hardware TM (e.g., Intel Haswell TSX). There is a draft specification for the interface to the TM at http://justingottschlich.com/tm-specification-for-c-v-1-1/, which is not too painful. You can also find use cases of gcc's TM from the TM community (see, for example, the application track papers from transact 2014: http://transact2014.cse.lehigh.edu).
The implementation itself is a bit complex, but that's what it takes to be correct. There's a lot of literature on the things that can go wrong, especially in a type-unsafe language like C or C++. GCC gets all of these things right. Really.
The second way is to use Java. You can either use DeuceSTM, which is very easy to extend (type safety makes TM implementation much easier!), or use Scala's Akka library for STM. I prefer Deuce, because it's easier to extend and easier to use (you just annotate a method as #Atomic, and Deuce's java agents do the rest).
The third way is to use Scala. I've not done much in this space, but researchers seem to love Akka. If you're affiliated with a parallel/distributed class, you might even be using Scala already.
So for the summer I decided that I may as well start learning algorithms before school starts. I've been told that the class is fairly fast paced, and that algorithms isn't something you should take lightly (I have a tendency to do this with all the course work during the semester lol).
The book we're going to use is this Algorithms (4th Edition).
Anyway, this is my problem.
I'm almost third way through the book, but I just realized what I was doing. For example, I would read and re-read the sections I don't quite understand. Then if I feel confident enough, I would try to reproduce the same algorithm in java from my head. But by doing this, my code looks almost exactly like the ones in the book..in java.
I can't say I'm just memorizing code after code--I do understand the concepts and they help me code these algorithms--but I feel like I'll only be able to implement these algorithms in java. I should note that I only know java at the moment.
tldr: I'm learning algorithms as if I'm learning to play the guitar--repetition after repetition. But by doing so I feel like I'm being more fixated that I'll only able to implement these in java. How exactly would you learn algorithms if the book you're using is language-specific?
Thanks in advance.
Don't Confuse Yourself
You're studying Java, so write them in Java. Especially if Java is your first language. Don't confuse yourself for now, as you are trying to learn 2 things at once: how to progam in Java, and how to progam. You're learning both a new language and a way of thinking. Don't do too much but adding another language to the sauce for now.
Diversify
Later on, or if you feel confident enough that you can take on another language simultaneously, then it would obviously be beneficial to learn another one and try to replicate the algorithms without looking at the book.
Reproduce and Extend
What we could recommend you is to look for derivates of the algorithms. Known variants, that have been documented, and where you could just read the description of the variant so you can try to implement it from the "base" version, without needing to read the book.
For instance, if your book introduced you to a linked list, you should be able to come up with the algorithm for a doubly-linked list or a circular linked list without reading more than a description of the desired outcome. Or there's something about the original concepts that you clearly misunderstood.
Try First, Read-On Later
I'd recommend you actually even try to implement the algorithms described in your book before they show them to you. The point of seeing Sedgewick's algorithm is to see a canonical implementation, which is considered a standard blueprint. If you just read the section leading up to the implementation (which hopefully is displayed first), then just sit down with the book, and try to figure out how you could do that. If you can't do that at all, then you're too far ahead in your book and should backtrack and start again from scratch.
Thing about algorithms, they're essentially language-agnostic. There's really nothing stopping you from doing Sedgewick's examples in C, Python or some other language.
If you really don't know any other languages, concentrate on Java. Sure, its a bit repetitious, but those bits will stick in your head in a good way and come test time, you'll be glad for the information.
You're in an interesting position right now, since the kind of thinking required to write programs is very different from normal thinking. Add to that the fact you're learning a whole new language with a different syntax, punctuation and the like. Practice really does make perfect, since there are many bits and pieces to remember.
Oh, if you want practice with algorithms, try out project euler, code kata and other challenge sites. These little challenges can help you familiarize yourself with the language as well as get comfortable with the type of thinking required.
First, congrats on taking your first steps on learning how to code. I would say that you are already ahead of your peers by starting to look ahead during the summer.
As far as your fears on only being able to implement algorithms in Java, you have already demonstrated that it will not be a problem for you. It sounds like you are passionate enough to get started early so you should have no problem implementing a solution in multiple languages. Additionally most of the languages with C/C++ (Java and C# to name a few) like syntax will be similar enough that you will be able to translate your knowledge seamlessly.
The best advice that I can give is to CODE, CODE, CODE!! Don't just read about the algorithms actually implement them.
You don't say how well you know the mathematics behind the algorithms. That will be key in determining your facility with the code.
Sedgewick's books are very good. I'd feel free to pick some and check out other books as well, like "Numerical Recipes" and "Numerical Methods That Work". See if another point of view can clarify for you.
If you don't feel like you're getting enough out of copying Java, see if you can translate them into another language, maybe Python or purely functional alternative. If you can do that, you'll know you've got it.
I would either try to learn another language to verify that you can actually port it to another language (javascript would be my vote because it is simple and useful on the front and backend) or write the algorithms out in pseudocode since that is more language agnostic. Most languages will have the code look pretty similar. The only thing to be very careful about is when you are relying on some aspect of the language (such as generics or iterators in java) which you may not be able to use in another language and that could leave a gap in your understanding.
Another way to verify that you actually understand the algorithm is to make slight changes in the problem and make sure that you can adjust the algorithm to still work. For example if it is a sorting algorithm then try to sort by several different attributes rather than just one, if it is a graph algorithm make the graph a digraph and see how things should change.
I'm learning algorithms as if I'm learning to play the
guitar--repetition after repetition.
Then you are not learning algorithms. You are learning repetition. Two different things. The usage of a programming language by an algorithms book is a secondary factor. It is just a vehicle of instruction, an implementation detail.
What you should be focusing is on understanding the structure, logic and mathematical characteristics of an algorithm (and possibly the data structure(s) associated with it.)
That's what your focus should be.
But by doing so I feel like I'm being more fixated that I'll only able
to implement these in java.
But that is because you are focusing on just how the algorithm is being coded (in Java in this particular case.) You are focusing on an implementation detail.
When you learn to drive, you don't focus on how you learn to drive a Honda Civic or a Nissan Maxima. You learn the essence of what driving is, the rules of thumbs, the necessary precautions and the laws governing driving a vehicle.
Same with learning algorithms. You don't learn "Algorithms in Java" no more than "Algorithms in Haskell". You learn Algorithms first and foremost, the vehicle (sans very specialized cases) is secondary.
You should be focusing on what the algorithm does, how and why. Questions like "how/why does it work?" and most importantly *"what are the performance characteristics?", those are the things you should be focusing on.
Every good algorithms book (Sedgewick's included) carry that message. That's what you should focus on. How you get to that re-focusing, that's a function of one's personal learning strategies.
How exactly would you learn algorithms if the book you're using is language-specific?
By not focusing on the language. Focus on the structure, focus on the data structures involved, the invariants, pre-conditions and post-conditions. Understand asymptotic behavior described in Big-O (or Big-Omicron), Little-O/Little-Omicron and Omega notations.
You are learning algorithms, not programming in Java via coding algorithms.
If you can't do this mental leap, it means you do not have sufficient practice or abstract analysis. It is not an insult, but an observation and an advice. Coding, the usage of a programming language is typically secondary to the mathematical analysis of computing, the focus of Computer Science (of which Algorithms is a part thereof.)
NOTE I've done Java for over 10 years, and though I like it for work, I strongly believe it is a poor tool for learning programming or CS topics.
One is better served by learning Algorithms with either A) a procedural, systems-level programming language like C or Ada, or a high-level pseudo-assembler simulator, or B) a functional language like Lisp or Haskell.
Object-Oriented features in pure/pseudo-pure OO languages simply get in the way.
Algorithms are mathematical structures with a nature descriptive of the how (operationally) and/or the what (mathematically). The former is perfectly suited for procedural programming, the later for functional programming.
I am student striving to improve my Java and general programming skills. I am familiar with some of the basic design patterns.
I have two books on my disposal "Head First to Design Patterns" and "Effective Java"(Josh Bloch), both aiming to promote good programming practices.
However I am confused as to "how should I read the texts so that it makes the most sense?"
From what I understand, design patterns are basics and effective java has some addon practices in addition to general design patterns.
What is the right way to look on the entire scenario?
You just have to build your knowledge up. Some of that is by reading good books, of which the Josh Bloch book is a very good book. Some of it is by taking on programming tasks in your spare time and keeping an eye on blogs and other websites such as stackoverflow. While some of it you will get through other practical experience i.e. at work.
These things take time and come with experience. Just enjoy the language and immerse yourself in it. Patterns aren't the be all and end all of the language - they're part of an effective toolkit though. It's better to understand the language fully, then you will see how design patterns can help solve common problems and often solve them better. But definitely don't get obsessed by patterns, until you know the language well enough. If fact don't get obsessed by patterns at all!
I'd say the Josh Bloch book is the best book to concentrate on 1st - it's excellent and introduces you to all the key concepts (common methods such as equals and hashcode, enums, concurrency) and also introduces patterns which you'll get day to day benefits from e.g. Builder pattern.
However I am confused as to "how should I read the texts so that it makes the most sense?"
My only advice is to avoid overkill and over-engineering. Some experienced OOP (and Java) programmers that learn design patters taking them as they were the Bible often create fuzzy and useless code. That seems contradictory, but if you think about it, it isn't.
When the goal of programming is no longer resolving a problem, producing useful tools, and having measurable results, but becomes the exposition of one's great sophistication, cleverness, and sometimes smartass egotism, then the code becomes just a bunch of tricky brainwave that is useless to anybody, excluding the one who wrote the code and feels good looking at his crazy "white elephant".
Many times I've seen useless applications of them. The most abused is the factory pattern, that in this moment is quite trendy and in fact put everywhere like a food dressing.
Many times I've seen proliferation of classes and interfaces, useless generalizations, which are pointless: in some cases I've seen packages with 2 useful classes, and 10 interfaces and/or abstract classes. I too feel the need of making the code "generic" and "reusable" but there has to be a limit.
Many times I've seen "religious wars" about these topics.
Read all the books you want, and take from them what you need. But after that, write reasonable code. Write useful stuff, that's the original reason why you started programming.
The two books complement each other. Effective Java is about best practices in writing java code. Design Patters describe a set of standard solutions to common software problems. The design patterns are language independent really, while Effective Java is obviously focused on the java language.
But my general approach is "always agree with Josh Bloch" ( unless Jon Skeet disagrees with him )
Programming patterns are best practices.
They should be used sparingly.
Every design pattern tries to handle some form of variation.
For example strategy pattern tackles a variation in implementation
Which is basically a fancy way of saying it is a plug and play method.
Patterns result in flexible code, but they create a large amount of extra complexity in the code.
Maybe you want to take the book Applied Java Patterns next to those books you already have.
It's written by Olav Maarsen and Stephen Stelting. Also the root of all patterns are the Gang of Four, you might want to use their book as a refference as well.
Design Pattern is not always good. Overuse of Design Pattern is bad.
Josh Bloch, however, is always right. Trust him and you are saved.
Forget Design Pattern. Josh Bloch's book gives you all the patterns you need.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
So do I really need to learn about them ? Isn't there an interesting way to learn about stacks, linked lists, heaps, etc ? I found it a boring subject.
**While posting this question it showed some warning. Am I not allowed to post such a question ? Admins please clarify and I will delete it :/
Warning :: The question you're asking appears subjective and is likely to be closed.
okay..I get it
So what is THE best way to learn them ? What book do I refer ? What website ?
It's compulsory to learn about data structures if you want to be a programmer. Data structures are your bread-and-butter - if you don't understand things like the behavior, uses, and run-time complexity ('big-O') of at least the basic structures (arrays, linked lists, stacks, queues, trees (binary / n-ary, self-balancing varietes), hash-tables, heaps, graphs) and the algorithms that run on them (insert / locate / delete), you won't know which is appropriate to use under what circumstances.
Every trade has its tools; these are ours. Data structures are the most basic underpinnings of almost any algorithm that you're going to learn. Unless you want to be a cargo cult programmer, you need to understand how they work.
Whether or not there are interesting ways to learn about them is a separate question entirely... :)
I would go even so far as to say that most of programming revolves around manipulating data structures, it is the foundation of computing after all: you get some data, you process it, you possibly give output. All the data usually reside in data structures and choosing inappropriate structures will have the bigger impact the bigger the project.
As you get more experience you will find that algorithms and datastructures are invaluable to your day-to-day development, and actually pretty interesting.
By learning about them now you will learn:
Which data structure is appropriate for which context, i.e. when to use a single-linked list, when to use a stack, when to use a queue, when to use a tree
Which algorithms are appropriate for which purpose, such as tree depth-first search or breadth-first search.
Space and time complexity of algorithms, for example why is a quicksort sometimes the best solution, and sometimes heapsort.
Overall it'll teach you the beginnings and fundamentals of computer science, even if you never have to implement a stack again you'll know the kind of thought and consideration that goes into it. If you then ever have to implement your OWN data structure (and chances are you will, quite often), you will know what to do and what not to do.
If you want to be a successful programmer, data structure is a must. How will you program if you don't know data structures and algorithms?
Whether you like it or not, all programming is built around data structures. You may never have to write one, but you will have to choose which one to use many times. It is not really a requirement to programming in general, but if you want to excel in the field, understanding of the basics is a must.
Anyone can build a shed without knowledge of materials or construction techniques. You can even work in a house placing bricks and mortar under the orders of someone else, but if you want to build a house yourself, you do need to understand materials and techniques.
Data structures are the programming materials. Algorithms are techniques. Will you use data structures? You will use the simplest ones in a daily basis, each so often you will need to solve a problem where an specific data structure is required, and while you may manage not to build your own bricks, you will need to understand whether you need bricks or a concrete wall for your purposes.
If you want some evidence for the importance of data structures, take a look at the Google hiring process. Whatever you think about Google as a company, there is no denying that they have some very good people working for them. Their interview process is setup to determine the candidates knowledge of data structures and algorithms. Because when it comes down to it, that is what is at the core of programming, no matter what language you are working in or what domain you are programming for.
If you are planning a career as a professional programmer, you need to know the fundamentals, not just how to crank out code that "works". Otherwise, your just playing.
Is it compulsory to learn about arithmetic to be an engineer?
If you take the attitude of "is it compulsory" with regards to any of the building blocks of programming languages, you're probably not cut out to be a coder. Regardless of "compulsory" or not, you should always be looking for new concepts to learn and seeing if it'll improve your coding style/standard.
But in answer to your question: yes.
I'd say it's compulsory at some point in your development to have a firm grasp. I'm not necessarily sure the standard Data Structures course is the best way to learn. Sometimes, the best way to learn them is "I have problem X. For some reason, it's taking my algorithm a long time to solve X. How can I make this faster?"
One book I'd highly recommend is Programming Pearls. It has some really good analyses, backed by a lot of examples of where the real-world motivations for the solutions came from. It presents the problems in an interesting manner, and never teaches by giving you a laundry list of data structures.
Yes 99% of books on Data Structures are boring and exercises contrived. They feel like they are just making up problems that server no practical purpose :(
This book is the one exception to the rule I've come across. You will have a naive but working RPG game by the end of the book:
Data Structures for Game Programmers
Read the above book and you will solve your chicken and egg problem and see you really can't do much without data structures after all.
Well, this may sound a little awkward but I wouldn't say it is COMPULSORY to learn Data Structures to be a - regular- developer. Seriously! Of courser if you study then hard it will give you a lot of insights and knowledge on several programming aspects and that's always good. But compulsory ... well, I think it is just too much. VERY GOOD would be enough.
Let me explain why. It is not that often, for today, to write Data Structures code because - let's face it - it would be RE-writing, RE-inventing what we already know for so many years! What I would say it is COMPULSORY is to study just the general theory of them and the APIs/libraries that are already commonly in use (and tested and optimized) like the Collections API in Java. You must know by heart the differences between a List and a Set (in Java for instance) and their capabilities and proper usage but you don't need to know exactly HOW they are implemented - checking every private method and attribute - to deal with most common, day to day, coding problems. You will do just fine without all the "guts" of Data Structures for the general stuff. We face different challenges now.
But don't get me wrong - neither think I am crazy or naive! Off course there are situations that you will need to implement yourself some sort of custom data structure (maybe your own BalancedBinaryTreeMap!). You got to be prepared for everything.
I am just arguing about being compulsory or not. Again, I don't think is compulsory but it is indeed very good.
Cheers.
I suppose you could learn programming without learning a whole lot about data structures or algorithms. To make an equivalent example, think of it like if a carpenter knew how to build things, but didn't know about measurements and such. Would he be able to get a career in carpentry? Possibly, but let's say he needed to know the exact material he would need to complete a project. He'd probably get fired because he doesn't know what sort of material or measurements to use.
So with data structures and algorithms, you can say it's the ability to give exact measurements of an application and knowing what sort of performance you'll get out it.
Like a musician learning scales, data structures are part of the tools of the software trade. Sure you can work as a programmer without the knowledge but you're handicapping yourself. If I'm interviewing two people for a position and one of them understands and uses structures and the other can't even explain what a stack is, my choice is pretty clear.
If you want to be judged to be a competent, employable programmer, you need to learn your craft.
Is it compulsory to learn them?
No, you can program without them, just as it's not compulsory to break your code into functions.
That being said, if you want to be an effective programmer that can write at least decent code without getting your car egged by your coworkers, you want to at least be able to make a decent selection of library classes.
Every programmer should understand the tradeoff between a LinkedList and an Array, or why binary searches and binary trees are useful for sorted data. This isn't just about performance - it's about correctness too since you can't just put anything into a tree set.
Does it mean that you need to know how to implement your own AVL tree, build super-smart data structures, etc.? Not necessarily. It's a matter of how much you want to know what's going on "beneath the hood" and whether your tasks necessitate it.
I'm not a big fan of deep data structure and algorithms questions in interviews because the vast majority of developers don't need to implement these things, just to use library stuff. I prefer to ask job related questions in interviews. However, accept that if you don't learn those things you would face a tougher battle to get other jobs.
It should be, yes...
No body should force you to learn anything you don't want to learn.
If you're the type of person that is compelled to be the best that he/she can be at what he/she does, and you love what you do for a living, you'll learn everything there is to know on your own accord.
#happysoul: You should ask yourself WHY learning data structures bore you. Also, it would help if you also identify what DOESN'T bore you.
If you at least love to learn about algorithms, I'm sure we can all suggest a perfect marriage of the two that would be exciting to learn!
My recommendation for best algorithm/data structure combo for the most fun learning experience: graphs.
Is there some resource for challenging multi-threading problems? Would like to pose these to interviewees if possible. Tired of asking the same wait-notify questions that everyone gets right these days, but can't visualise a real scenario where multi-threading was employed.
The problem is that concurrent programming is a difficult topic. If you (the interviewer) are not fully on top of it, it will be difficult for you to tell if the interviewee knows their stuff. It is very easy to come up with solutions to concurrency problems that have subtle flaws. Conversely, it is unfair on candidates1 if you reject them because you think their answers are wrong when they are actually correct.
1 - and bad for your organisation. If the candidate actually knows more about multi-threading than you, then you arguably need to employ him. Other factors being equal, of course.
Java Concurrency In Practice. I like to know if candidate understand data race, CAS, Michael Scott Queue and other concurrent data structures and how concurrent thread safety is important with growing number of cores.
As multithreading is hard (as others have pointed out) I would suggest having this in an actual programming session where the potential employee is given a programming problem preferrably based on something that has actually happened along with one of your experienced programmers so you can actually SEE how the problem was attempted solved, and the experienced programmer can evaluate what happened.
Must not be too complex, but complex enough that your expereinced programmer get enough information.
Well, if you want to have fun with the poor sap, ask him about Dekker's Algorithm (and Peterson's variation thereof). If you're feeling nasty, ask him if he has ever used either one on real multiprocessor hardware.
If you feel extra-nasty, ask him to show you a technique suitable for lock-free true concurrent single-reader single-writer unidirectional communications, between two processors with shared memory, in which the only atomic operations are single-word reads and writes. There is no read-modify-write instruction, on either side, and the processor architectures need not be the same. (Yes, such a technique exists.)
I wouldn't ask too specific/detailed questions. But the above mentioned book 'Concurrency in practice' is a good helper. Just go there chapter-wise and read out the pin-points, e.g.:
Explain difference between mutable/immutable
What does it mean to share data in concurrency setup
What problems do you solve with concurrency
etc.
First you
try to get the real scenario
of it, and then ask job seekers.
For this you should pose a questions like what is real scenario for multithreading?
Hope it will help you.
I got one in a interview recently. Get the candidate to write a Servlet that implements an accurate in memory hit counter indexed by URL (to serve a javascript style hit counter on a number of web page). Try it for yourself, it's not as easy as it sounds. The solution is a cut down implementation of the Memoizer pattern from Concurrency in Practice.