DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-6 are rejected under 35 U.S.C. 103 as being unpatentable over Exelsson et al. (US Patent Application Publication No. 2018/0129363) in view of Ma et al. (US Patent Application Publication No. 2017/0286764).
Regarding claim 1, Elexsson discloses a method comprising [see abstract; a computing device configured to display visual representations of digital notes and one or more group images representing one or more groups on a user interface, where each group may include one or more digital notes. The computing device is further configured to receive one or more user inputs via the user interface and change the compositions of the groups based on the received user inputs]; rotating the plane for each of the notes [see par. 0065; a free form group image, which represents the spatial positions of the physical notes. The computing device can generate group images with different notes alignments and arranges, either automatically or in response to user input. For example, is a fitted group image; is a group image where the individual visual representations of digital notes are snapped to grid; and is a group image that is generated by automatic layout, for example. In some cases, in the group image of the spatial positions of the digital notes within the grid can correspond to the spatial positions of the physical notes.]; and converting the notes to corresponding digital notes [see par. 0008, 0068; a “double tap” on a note create a new copy of the note which can be clicked and dragged to a new group. Double tap on a group can create a copy of the group. In some cases, grouping achieved by touching and dragging the group boundary to include new notes. Alternatively, by clicking and dragging a bounding rectangle or drawing a loop/circle around notes using stimuli (e.g., finger, stylus, etc.) on a touch sensitive device, a group can be created, new notes are captured and presented on the GUI and the new notes can be added to the existing groups, for example by connecting subsequent loops/circles/rectangles around new notes, to the group images]; however, Exelsson fails to explicitly teach receiving a digital 3D model of notes and finding a plane and anchors for each of the notes; computing a transformation matrix, based upon the anchors, to find corners of the notes.
Ma discloses receiving a digital 3D model of notes [see par. 0104; For forms, again field designators are exemplary feature zones suitable for locating identifying features, as well as lines (particularly intersecting lines or lines forming a vertex), boxes] ; finding a plane and anchors for each of the notes; computing a transformation matrix, based upon the anchors, to find corners of the notes [see par. 0133, 0139, 0141, 0234; Key point selection includes finding pixels in an image that have distinctive visual features. These pixels with distinctive features are at positions where image intensities change rapidly, such as corners, stars, etc. Theoretically speaking, every pixel in an image can be selected as a key point and the matching process selects the reference image with the maximum number of matching points as the best match, and an affine or homograph transform is estimated with the best match to reconstruct the image and/or video data in a three-dimensional coordinate system.]
It would have been obvious to one of an ordinary skill in the art, having the teachings of Exelsson and Ma before the affective filing date of the claimed invention to modify, organizing digital notes on a user interface of Exelsson to include three dimensional reconstruction of objects in image and video data, as taught by Ma, with a reasonable expectation of success. The motivation for a computer user may create digital notes and attach the digital notes to an electronic document, a desktop, or an electronic workspace presented by the computing environment and manipulate the notes, allowing the notes to be created, deleted, edited, saved, and selectively viewed. In this way, the software programs provide a virtual representation of notes and allow an individual to utilize the digital notes in a manner similar to physical notes that he or she may use on a daily basis and to improve the accuracy of digitizing notes that appear in 3D scanned views.
Regarding claim 2, Ma discloses further comprising: re-calculating the corners according to a projection matching a plurality of camera angles; computing a score for the corners based upon the re-calculating at each of the camera angles; and selecting a projection angle for the corners based upon the computing and, based upon an aspect ratio of the digital note at the selected projection angle, calculating a size for the digital note [see par. 0131, 0142, 0235; an “oriented Features from Accelerated Segment Test (FAST) and rotated BRIEF” (ORB) algorithm may be employed to enhance descriptors with orientation information. After getting the patch orientations, each descriptor is normalized by rotating the image patch with the estimated rotation angle. The foregoing three-step process is repeated in an iterative fashion to re-sample the key points and estimate a new homograph transform. In one embodiment, the number of iterations performed may be in a range from about 10.sup.2-10.sup.3 iterations. After the iterative identification of key points is complete, the largest inlier set is retained, and an affine or homograph transform is re-estimated based on the retained inlier set].
Regarding claim 3, Ma discloses wherein re-calculating step comprises using a maximum camera angle and a minimum camera angle [see par. 0235; the bottom of an invoice containing a deposit slip. In a situation like this the bottom, left and right sides of the document can be correctly determined and used to estimate pitch and roll; these angles together with the focal distance can be used to rectangularize the visible part of the document].
Regarding claim 4, Ma discloses wherein re-calculating further comprises re-calculating the corners at increments between the plurality of camera angles [see par. 0194, 0231; The approach also uses the known focal distance of the camera in the calculations of the coordinate transform, but if all four corners have been found and there are three independent angles, then the same criterion and a slightly more complex gradient descent procedure can be used to estimate the focal distance in addition to pitch and roll—this may be useful for server-based processing, when incoming pictures may or may not have any information about what camera they were taken with.
Regarding claim 5, Ma discloses wherein re-calculating further comprises: calculating a pixels per inch value for the selected size; and determining a difference between the calculated pixels per inch value and an area of a physical note corresponding with the selected size [see par. 0194].
Regarding claim 6, Exlesson discloses wherein comprises rotating the plane of each note such that the note appears captured from a front non- angled view [see figures 5d-5E].
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure (See PTO-892).
Shinkichi (US 2018/0268932) discloses To provide an electronic sticky note system with which it is possible to restrict users who are displaying electronic sticky notes and sharing information, and to display only a minimum of electronic sticky notes, as well as to enable the setting of displays for multiple cases requiring the display of an electronic sticky note to be performed all at once. [Solution] An electronic sticky note system equipped with: a case ID management means that manages IDs for multiple cases; a document type management means that manages multiple document types constituting case records; a case-specific/document-type-specific recording means that manages records for each document type of each case; a staff ID management means that manages the IDs of multiple staff; an electronic sticky note creation means that creates and records electronic sticky note content together with a display format; an electronic sticky note display condition setting means that sets and records conditions for displaying the electronic sticky notes; and an electronic sticky note display authorization determination means that determines whether an electronic sticky note satisfies the “conditions for displaying” when case records are displayed.
A reference to specific paragraphs, columns, pages, or figures in a cited prior art reference is not limited to preferred embodiments or any specific examples. It is well settled that a prior art reference, in its entirety, must be considered for all that it expressly teaches and fairly suggests to one having ordinary skill in the art. Stated differently, a prior art disclosure reading on a limitation of Applicant's claim cannot be ignored on the ground that other embodiments disclosed were instead cited. Therefore, the Examiner's citation to a specific portion of a single prior art reference is not intended to exclusively dictate, but rather, to demonstrate an exemplary disclosure commensurate with the specific limitations being addressed. In re Heck, 699 F.2d 1331, 1332-33,216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006,1009, 158 USPQ 275, 277 (CCPA 1968)). In re: Upsher-Smith Labs. v. Pamlab, LLC, 412 F.3d 1319, 1323, 75 USPQ2d 1213, 1215 (Fed. Cir. 2005); In re Fritch, 972 F.2d 1260, 1264, 23 USPQ2d 1780, 1782 (Fed. Cir. 1992); Merck & Co. v. Biocraft Labs., Inc., 874 F.2d 804, 807, 10 USPQ2d 1843, 1846 (Fed. Cir. 1989); In re Fracalossi, 681 F.2d 792,794 n.1,215 USPQ 569, 570 n.1 (CCPA 1982); In re Lamberti, 545 F.2d 747, 750, 192 USPQ 278, 280 (CCPA 1976); In re Bozek, 416 F.2d 1385, 1390, 163 USPQ 545, 549 (CCPA 1969).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CAO H NGUYEN whose telephone number is (571)272-4053. The examiner can normally be reached on Mon-Fri 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached on 571-272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CAO H NGUYEN/Primary Examiner, Art Unit 2171
Prosecution Insights